Device for harvesting bacterial colony and method therefor

ABSTRACT

When multiple kinds of bacterial colonies are present in a petri dish and, for example, a drug tolerance is to be measured, harvesting of mixed colonies of different types of bacteria makes it impossible to accurately determine the drug tolerance. Also, it is required to improve the throughput of a device for harvesting a bacterial colony. From images illuminated from multiple directions, isolating bacterial colonies are automatically extracted. Next, the image feature amounts are calculated from the multiple images that are illuminated from multiple directions and colonies are grouped depending on the feature amounts. Then, bacterial colonies to be harvested are determined based on the results of the grouping.

TECHNICAL FIELD

The present invention relates to a device for harvesting a bacterialcolony cultured in a petri dish, and particularly, to a device forharvesting a bacterial colony and a method thereof that image an opticalimage of a petri dish, automatically classify a plurality of kinds ofbacterial colonies cultured in the petri dish based on the imaged imagefor each same bacterial species, harvest a plurality of bacterialcolonies for each bacterial species, and identify harvested bacteria.

Further, the present invention relates to a device for harvesting abacterial colony and a method thereof, and particularly, to apreprocessing apparatus that cultures bacteria in a specimen, selects abacteria colony, and adjusts a bacterial liquid of the selectedbacterial colony in order to perform identification and drugsusceptibility test of bacteria included in the specimen.

In addition, the present invention relates to a device for harvesting abacterial colony and a method thereof, and particularly, a preprocessingapparatus for harvesting a bacterial colony for an apparatus and amethod for analyzing bacteria that test identified drug susceptibility.

BACKGROUND ART

In testing bacteria with respect to a specimen, a method of culturingthe bacteria in a culture medium within a petri dish and harvesting thebacteria has been used. In this method, in general, a suspension of thebacteria is smeared to the culture medium and the bacteria are culturedin a constant-temperature bath and thereafter, optically visuallyobserved, and a target bacterial colony is harvested. The harvestedbacteria are used for the purpose of testing antibiotic tolerance. Amethod of testing the antibiotic tolerance of the bacteria is disclosedin detail in Patent Literature 1. In a minimum inhibitory concentration(MIC) method, a diluted liquid of an antibiotic substance is injectedinto a liquid medium or a solid medium, the harvested bacteria colonythe medium with the harvested bacterial colony, and a growing conditionof the bacteria is observed by culturing the bacteria. A culturingcondition is observed in a plurality of media by variously changing theconcentration of the diluted liquid of the antibiotic substance, therebymaking it possible to acquire a minimum antibiotic substanceconcentration which cannot be grown and an appropriate drugconcentration capable of reducing a burden to a patient can bedetermined.

Meanwhile, Patent Literature 2 discloses a bacterial colony transferringapparatus in which a bacterial colony within a petri dish isphotographed with a television camera, a tester visually verifies animage projected on a monitor and indicates a bacterial colony to beharvested to automatically move a pick-up to the position of thebacterial colony, harvests the bacterial colony, and transfers theharvested bacterial colony to a test tube.

Further, Patent Literature 3 discloses a method of analyzing a result ofharvesting using filtering after liquefying a specimen with a Fouriertransform infrared spectrophotometer (FT-IR) or a microscopic infraredspectrophotometer and comparing a waveform of spectral absorbance with awaveform registered in a database to identify the waveform.

A microbial analysis apparatus is used to, in testing a specimen in amedical practice, identify a species of a microorganism included in thespecimen and measure susceptibility of the microorganism to anantibiotic. Establishing a therapeutic measure by determining a kind anda concentration of a drug which is effective with respect to themicroorganism detected from the specimen is very important to performappropriate antimicrobial therapy.

Herein, one example of a flow of testing a specimen will be describedbelow. First, collected specimens (for example, blood, sputum, feces,and the like) are dyed and observed with a microscope. When a positivereaction is observed, the specimens are moved onto a medium (forexample, agar, and the like) of a petri dish and the petri dish isinputted into an incubator. One day after, when the petri dish isextracted from the incubator, a plurality of kinds of bacterial coloniesof microorganisms are generated on the medium. One kind of microorganismis selected and harvested among the plurality of kinds of bacterialcolonies and moved to the test tube and bacterial liquids having aplurality of kinds of concentrations are adjusted by adjusting theconcentration thereof with normal saline. The bacterial liquids adjustedto the plurality of kinds of concentrations are inputted into amicroorganism testing apparatus, and identification of bacteria and drugsusceptibility are examined.

In order to rapidly and accurately test the specimen, a microorganismanalysis apparatus that automates each work is proposed. For example,Patent Literature 4 discloses an automatic analysis apparatus ofbacteria. The automatic analysis apparatus of bacteria disclosed inPatent Literature 1 includes a test liquid preparing unit, apre-incubation, a top agar dispensing unit, a plating unit, a platereceiving unit, a bacterial colony counting unit, a data processingunit, a test tube transferring means, and a plate receiving racktransferring means and can automatically perform a series of analysisoperations of counting the number of bacterial colonies with highprecision in preparing the test liquid.

Patent Literature 5 discloses a microorganism screening apparatusincluding an image processing apparatus detecting a position of abacterial colony, an elevator unit transferring a work with a rotatingoperation mechanism and an ascending and descending operation mechanism,a harvesting/inoculating unit that performs harvesting and inoculatingwhile determining harvesting and inoculating positions by operating arobot, a work transporting unit moving and transporting the work, acover opening/closing unit controlling opening/closing a cover of thework, a liquid dispensing unit performing dispensing of a liquid medium,an agar dispensing unit dispensing an agar medium, and a host computerunit performing control operations such as management of data ofmicroorganism resources, a command of an operation mode, and the like.The microorganism screening apparatus disclosed in Patent Literature 5aims at automating various operations required for screening a soilmicroorganism and further, improving reliability by completely managinga strain and data.

Patent Literature 6 discloses a method and an apparatus for testing amicroorganism colony. The method and apparatus for testing themicroorganism colony disclosed in Patent Literature 6 may compare animage of a petri dish, an image before a bacterial colony is generated,and an image after the bacterial colony is generated, removecontamination of a photographing system or a residue on a medium fromthe image after the bacterial colony is generated, and accurately countthe number of bacterial colonies. It is described that, by combining amark printed on the petri dish with each image in advance, the imagesmay be accurately compared with each other.

Patent Literature 7 discloses a method and an apparatus for collecting abacteria colony. The method and apparatus for collecting a bacteriacolony disclosed in Patent Literature 7 aims at installing a positionreference means of a bacterial colony, determining an absolute spatialposition of the bacterial colony, and accurately determining a place ofa harvested bacterial colony.

In infectious disease therapy, establishing a therapy measure byidentifying mastoiditis, rapidly measures susceptibility to antibiotic,and determining an effective drug is important for appropriateantimicrobial therapy. In general, a presented specimen is applied to amedium and cultured, bacterial suspension is prepared by harvesting thegenerated bacterial colony and suspending the harvested bacterial colonyto normal saline, and the bacterial suspension is inoculated into ameasuring device of an identification and drug susceptibility testingapparatus. In identification and susceptibility testing, it is requiredto make the amount of bacteria inoculated into the device constant atpredetermined concentration with accuracy and high reproducibility, inorder to acquire an accurate result. In most cases, in order to acquirea predetermined amount of bacteria, a plurality of same kinds ofbacterial colonies are selected and harvested among bacterial coloniesgrown on a petri dish, suspended to a liquid such as normal salinewithin one vessel, and prepared with a predetermined concentration (thenumber of bacteria) by measuring turbidity or opacity. Kinds of mixedbacterial colonies should be the same as each other and a plurality ofkinds of bacterial colonies are generally grown on the same petri dish,and a high-level technique is required of a laboratory technician inselecting the same kind of bacterial colonies among them. Further, sincethe same kind of bacterial colonies are selected from various large andsmall-sized bacterial colonies, it is required that turbidity is firstadjusted and thereafter, the bacterial colonies are added or normalsaline for dilution is added in order to acquire predeterminedturbidity, and as a result, it is complicated and it takes some time.Therefore, the method and apparatus for collecting a bacteria colony arenot appropriate to mass processing. As a related art to make the amountof harvested bacteria per one time constantly, a simple kit has beencommercialized, which can acquire a predetermined concentration ofbacterial liquid by harvesting a predetermined amount of bacteria andsuspending the harvested bacteria to a predetermined amount of normalsaline. For example, a method disclosed in Patent Literature 8 disclosesan example capable of collecting a predetermined amount of bacteria byusing a simple stick body with a groove. However, in this method, aconcentration and the amount of bacterial liquid acquired are limited,and as a result, only a small amount of bacterial liquid with acomparatively low concentration range can be acquired.

Meanwhile, Patent Literature 2 discloses, as a bacterial colonytransferring apparatus, a method in which the bacterial colony within apetri dish is photographed with a television camera, the laboratorytechnician visually verifies an image projected on a monitor and selectsand indicates a bacterial colony to be harvested to automatically move aharvesting tool to the position of the bacterial colony according to theindication, harvest the bacterial colony, and transfer the harvestedbacterial colony to a test tube. However, in this method, the imagewhich the laboratory technician photographs by using the televisioncamera is verified through the monitor to select the harvested bacterialcolony one by one, and as a result, even though the apparatus is used,an effect of rapidity is slight.

Similarly, Patent Literature 9 and Patent Literature 10, as thebacterial colony transferring apparatus, discloses a method in which alaboratory technician verifies an image photographed by a televisioncamera through a monitor, inputs a condition such as the size of abacterial colony to be selected or indicates a bacterial colony to beexcluded, and as a result, a bacterial colony to be harvested isautomatically transferred to a new medium. In this method, the bacterialcolonies grown on the petri dish are basically all the same bacterialspecies and all bacterial colonies other than an exception grown bycontamination and the like are transferred, and since the apparatus isunmanly operated by inputting only a condition, significant power savingis achieved. However, by harvesting the bacterial colony one by one fromthe petri dish before transferring and transferring the bacterial colonyto a medium of a new petri dish one by one, the bacterial colony neednot be mixed, and as a result, a type of the bacterial colony is notdiscriminated.

In preparing the bacterial liquid for identification testing or drugsusceptibility testing intended in the present invention, a plurality ofsame kind of bacteria needs to be harvested from one petri dish anddifferent kinds of bacteria need to be excluded and only a single kindof bacteria needs to be selected from the image acquired by thetelevision camera. However, since a comprehensive feature of thebacterial colony cannot be normally extracted by only one image acquiredby using the television camera, an error may occur when the bacterialcolony is selected. Meanwhile, a bacterial colony counter that measuresthe number of bacterial colonies including bacterial colonies on theagar medium as well as the surface of the petri dish has been availablein the market. This processes a 2D image photographed by the camera andmeasures the number of bacterial colonies having a size larger than asize suitable for the condition and similarly, cannot discriminatedifferent kinds of bacterial colonies. In the above example, theacquired image is based on plane information and the size of thebacterial colony can be measured, but information on a shape or a heightdirection of the bacterial colony cannot be acquired.

CITATION LIST Patent Literature

-   Patent Literature 1: Japanese Patent Application Laid-Open    Publication No. 63-32477-   Patent Literature 2: Japanese Patent Application Laid-Open    Publication No. 59-11173-   Patent Literature 3: Japanese Patent Application Laid-Open    Publication No. 2005-55180-   Patent Literature 4: Japanese Patent Application Laid-Open    Publication No. 2002-98704-   Patent Literature 5: Japanese Patent Application Laid-Open    Publication No. 9-47281-   Patent Literature 6: Japanese Patent Application Laid-Open    Publication No. 2005-143425-   Patent Literature 7: Japanese Patent Application Laid-Open    Publication No. 2000-171360-   Patent Literature 8: Japanese Patent Application Laid-Open    Publication No. 58-201976-   Patent Literature 9: Japanese Patent Application Laid-Open    Publication No. 62-25348-   Patent Literature 10: Japanese Patent Application Laid-Open    Publication No. 62-65700-   Patent Literature 11: Japanese Patent Application Laid-Open    Publication No. 2000-78999-   Patent Literature 12: Japanese Patent Application Laid-Open    Publication No. 7-306023

SUMMARY OF INVENTION Technical Problem

In order to implement an MIC method, since bacteria need to be culturedin a plurality of media in which the concentration of the antibioticsubstance is changed, a plurality of bacterial colonies cultured in thepetri dish need to be harvested. When a relatively small bacterialcolony is not also harvested, a long time is required until thebacterial colony is harvested after being cultured in the petri dish andculturing within a short time is difficult.

Further, in the MIC method, when all kinds of harvested bacterialcolonies are not the same as each other, accurate tolerance of theantibiotic substance cannot be tested, but up to now, the tolerancedepends on the visual result of the tester, such that traceabilitycannot be acquired. In addition, visual testing may be differentaccording to a technique of the tester, such that only a few testerscannot but test the tolerance to acquire stable harvesting and aplurality of specimens cannot be processed.

Meanwhile, as a bacterial colony transferring apparatus disclosed inPatent Literature 2, in a method in which the bacterial colony withinthe petri dish is imaged with a television camera, the tester verifiesan image projected on a monitor and indicates a bacterial colony to beharvested to automatically move a pick-up to the position of thebacterial colony, harvest the bacterial colony, and transfer theharvested bacterial colony to a test tube, since the tester selects theharvested bacterial colonies one by one by verifying the televisioncamera, a man hour of the tester is not so reduced. Further, in oneimage acquired by the television camera, an error occurs in selectingthe bacterial colony in some cases. For example, an appearance of anykind of bacteria is apparent by transmitted illumination and otherbacteria tend to be apparent by illumination from the top. When the MICmethod is used, a plurality of same kind of bacteria need to beharvested, and as a result, different kinds of bacteria need to beexcluded and only a single kind of bacterium needs to be selected fromthe image acquired through the television camera and the bacterialcolony cannot be discriminated with sufficient precision by only theimage of the television camera.

Further, in the method disclosed in Patent Literature 3 in which afterthe specimen is liquefied, a bacteria collection result by filtering isanalyzed by FT-IR and the waveform of spectral absorbance is comparedwith the waveform registered in a database to identify the waveform,culturing for using the MIC method after analysis through the FT-IR isimpossible.

A first object of the present invention is to provide a device forharvesting a bacterial colony and a method thereof that can specifybacterial colonies of the same kind to be harvested among bacterialcolonies group and stably harvest the plurality of specified bacterialcolonies by solving a known technical problem.

In general, culturing the bacterial colony requires a time over onenight, but hundreds or more of petri dishes may be tested according to ascale of a test room. Culturing the bacterial colony in the hundreds ofpetri dishes one by one, selecting an appropriate bacterial colony fromthe plurality of bacterial colonies grown in the petri dish andharvesting the appropriate bacterial colony by the harvesting tool, anddiluting the petri dish with normal saline or adding the bacterialcolony, and adjusting the bacterial suspension while monitoring whetherthe bacterial colony has desired turbidity require a lot of time andmanpower cost. Therefore, development of an apparatus that can automatethe work from the growth of the bacterial colony to the adjustment ofthe bacterial suspension is required.

The analysis apparatus disclosed in Patent Literature 4 does not havethe means for selecting the bacterial colony. Further, in PatentLiterature 5, Patent Literature 6, and Patent Literature 7, sinceposition detection and harvesting of the bacterial colony are performedwith one image, selection and harvesting of the bacterial colony cannotbe performed in parallel, such that it is difficult to improve aprocessing speed.

Therefore, a second object of the present invention is to provide adevice for harvesting a bacterial colony and a method thereof that canperform selection and harvesting of the bacterial colony in parallel andimprove the processing speed.

Further, an object to be achieved in the present invention is toestimate height-direction information of collected bacterial colonies,automatically calculate the volume of the bacterial colony, andautomatically harvest the required number of bacterial colonies withrespect to individual bacterial colonies in order to collect apredetermined quantity of bacterial colonies required for anidentification and susceptibility test after discriminating thebacterial colonies for each same kind by the image of the petri dishacquired through imaging, and determining and selecting the bacterialcolony to be harvested.

Further, a method and an apparatus for automatically preparing abacterial liquid having predetermined turbidity are provided afterautomatic harvesting according to acquired information by adopting thismethod, such that an apparatus and a method for analyzing amicroorganism with rapidity and high power saving effect are provided,thereby reducing the manpower cost of a bacteria test room.

Solution to Problem

In order to achieve the objects, in the present invention, a device forharvesting a bacterial colony is configured to include: an upperillumination means illuminating a bacteria colony cultured on a culturemedium received in a first vessel which is optically transparent, fromthe top; a transmitted illumination means illuminating the bacterialcolony by transmitting the optically transparent vessel and the culturemedium with illumination light; an imaging means imaging the bacterialcolony sequentially illuminated by the upper illumination means and thetransmitted illumination means; an image processing means extracting animage of a bacterial colony to be harvested by processing an image ofthe bacterial colony illuminated by the upper illumination means and animage of the bacterial colony illuminated by the transmittedillumination means, which are imaged by the imaging means; and aharvesting means harvesting the bacterial colony extracted by the imageprocessing means from the culture medium and moving the harvestedbacterial colony to a second vessel.

Further, a method for harvesting a bacterial colony according to thepresent invention includes: illuminating a bacteria colony cultured on aculture medium received in a first vessel which is opticallytransparent, from upper side and imaging the bacterial colony to acquirean upper side illumination image of the bacterial colony; illuminatingthe bacterial colony by transmitting the optically transparent vesseland the culture medium with illumination light and imaging the bacterialcolony to acquire a transmitted light illumination image of thebacterial colony; extracting an image of a bacterial colony to beharvested by processing the upper side illumination image and thetransmitted light illumination image; and harvesting a bacterial colonycorresponding to the extracted image from the culture medium and movingthe harvested bacterial colony to a second vessel.

Further, in order to achieve the object, the present invention providesa device for harvesting a bacterial colony, including: a specimen stageon which a petri dish is loaded; an imaging stage imaging the inside ofthe petri dish from a top surface of the petri dish and having a storagemeans storing the image; a selection means automatically detectingbacterial colonies of microorganisms in the petri dish and selectingsome of the detected bacterial colonies; an image display meansdisplaying the image in the petri dish from the top surface of the petridish; a comparison means reading the image imaged with the imaging stagefrom the storage means and comparing the read image with the imagedisplayed with the image display means; a harvesting means including adriving unit for harvesting the bacterial colonies of the microorganismsin the petri dish; a harvesting stage including the harvesting means,the image display means, the comparison means, and the selection means;a first buffer station storing a plurality of petri dishes beforeimaging; a second buffer station storing a plurality of petri dishesafter imaging; a first transportation means sequentially supplying petridishes before imaging from the first buffer station to the imagingstage; a second transportation means sequentially supplying petri dishesafter imaging from the imaging stage to the second buffer station; athird transportation means sequentially supplying petri dishes beforeharvesting from the second buffer station to the harvesting stage; and acontrol unit controlling the imaging stage, the harvesting stage, thefirst buffer station, the second buffer station, the firsttransportation means, the second transportation means, and the thirdtransportation means, wherein the selection means automatically selectsa bacterial body to be harvested from an appearance feature of thebacterial colony in the imaged petri dish with respect to each petridish.

Further, in order to achieve the object, the present invention providesa device for harvesting a bacterial colony, including: a specimen stageon which a petri dish is loaded; an imaging stage imaging the inside ofthe petri dish from a top surface of the petri dish and having a storagemeans storing the image; a selection means automatically detectingbacterial colonies of microorganisms in the petri dish and selectingsome of the detected bacterial colonies; an image display meansdisplaying the image in the petri dish from the top surface of the petridish; a comparison means reading the image imaged with the imaging stagefrom the storage means and comparing the read image with the imagedisplayed with the image display means; a harvesting means including adriving unit for harvesting the bacterial colonies of the microorganismsin the petri dish; a harvesting stage including the harvesting means,the image display means, the comparison means, and the selection means;a first buffer station storing a plurality of petri dishes beforeimaging; a second buffer station storing a plurality of petri dishesafter imaging; a first transportation means sequentially supplying petridishes before imaging from the first buffer station to the imagingstage; a second transportation means sequentially supplying petri dishesafter imaging from the imaging stage to the second buffer station; acontrol unit controlling the imaging stage, the harvesting stage, thefirst buffer station, the second buffer station, the firsttransportation means, and the second transportation means, wherein theimaging stage and the harvesting stage are positioned within a commonarea.

The present invention includes a camera, a base on which the bacterialcamera on the culture medium to be imaged is mounted, a plurality offirst illumination units radiating light to the bacterial colony fromdifferent positions on the top, and a second illumination unitsilluminating the culture medium from the bottom, which is positioned atan opposite side to the camera with respect to the base.

Advantageous Effects of Invention

According to the present invention, by using the image acquired byilluminating the petri dish from the top and the image acquired bytransmitting and illuminating the petri dish from the bottom, the imageof the bacterial colony may be separated from an image of a character ora symbol written or formed on the petri dish to be detected and thebacterial colony species may be classified and accurately harvested byimage processing.

Further, illumination from upper side is performed in a plurality ofelevation angle/direction angle directions and an illumination imagefrom multiple directions is used to make almost every bacterial colonyon the medium apparent, the accurate position of the bacterial colony isfeasible, and an appearance feature of the bacterial colony iscalculated from the illumination image from the multiple directions,thereby achieving accurate grouping. In addition, a plurality of imagesilluminated and imaged from different directions are displayed on thescreen, such that it is easy for even a user to visually verifydifferent kinds of bacteria.

Further, according to the present invention, selection and harvesting ofthe bacterial colony can be performed in parallel and the throughput intesting the microorganism can be improved.

In addition, according to the present invention, since the heightinformation of the bacterial colony is estimated and the volume of thebacterial colony is calculated from a plurality of first illuminationunits radiating light to the bacterial colony from different positionsfrom upper side and a second illumination unit illuminating the mediumfrom the bottom, which is positioned at an opposite side to the camerawith respect to a base, a quantity required to acquire a bacterialliquid having predetermined concentration (turbidity) can be estimated.As a result, accuracy of the quantity of collected bacterial coloniescan be improved, the work such as addition or dilution of the bacterialcolony can be remarkably reduced, and the bacterial liquid having thepredetermined concentration can be rapidly prepared.

Consequently, a time required for the entire bacterial test can beshortened.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram showing a schematic configuration of an entiresystem according to a first embodiment.

FIG. 2A is a front view showing a schematic configuration of anillumination system according to the first embodiment.

FIG. 2B is a plan view showing the schematic configuration of theillumination system according to the first embodiment.

FIG. 3A is a front view showing a schematic configuration of anotherexample of the illumination system according to the first embodiment.

FIG. 3B is a plan view showing the schematic configuration of anotherexample of the illumination system according to the first embodiment.

FIG. 4A is a front view showing a schematic configuration of yet anotherexample of the illumination system according to the first embodiment.

FIG. 4B is a plan view showing the schematic configuration of yetanother example of the illumination system according to the firstembodiment.

FIG. 5A is a diagram showing a relationship between an illuminationcondition of an imaged image of a bacterial colony by a detection systemand a shape of the bacterial colony according to the first embodiment.

FIG. 5B is a cross-sectional view of the bacterial colony describing arelationship of incident light and reflected light to the bacterialcolony by the detection system and a normal line direction on thesurface of the bacterial colony according to the first embodiment.

FIG. 6 is the imaged image of the bacterial colony by the detectionsystem according to the first embodiment.

FIG. 7 is a graph showing a luminosity profile of the imaged image ofthe bacterial colony by the detection system according to the firstembodiment.

FIG. 8A is a cross-sectional view of a petri dish showing (a) an imagedimage by transmitted illumination and (b) a transmitted illuminationdirection in the first embodiment.

FIG. 8B is a cross-sectional view of the petri dish showing (a) animaged image by low-angle illumination and (b) a low-angle illuminationdirection in the first embodiment.

FIG. 8C is a cross-sectional view of the petri dish showing (a) animaged image by high-angle illumination and (b) a high-angleillumination direction in the first embodiment.

FIG. 9 is a flowchart showing a sequence of extracting a bacterialcolony candidate area of bacteria in the first embodiment.

FIG. 10 is a flowchart showing a sequence of extracting an image featureamount of a bacterial colony of bacteria in the first embodiment.

FIG. 11 is a graph showing a luminosity profile of the bacterial colonyof the bacteria in the first embodiment.

FIGS. 12( a), 12(b), and 12(c) are a database describing aclassification method of the bacterial colony of the bacteria in thefirst embodiment, a table showing one example of an index of thedatabase, and a graph showing the distribution of feature vectors,respectively.

FIG. 13A is a graph showing one example of the distribution in a featureamount space of the bacterial colony of the bacteria in the firstembodiment.

FIG. 13B is an image of the petri dish in which the bacterial colony ofthe bacteria is distributed in the first embodiment.

FIG. 14 is a block diagram showing a schematic configuration of aninfrared spectroscopic detection system in the first embodiment.

FIG. 15 is a graph showing a relationship between a wave number andlight absorbance of light in the first embodiment.

FIG. 16 is a block diagram showing a schematic configuration of anentire system according to a second embodiment.

FIG. 17 shows an embodiment of the illumination system of the presentinvention.

FIG. 18 is a schematic diagram showing one example of a device forharvesting a bacterial colony according to the present invention.

FIG. 19 is a plane schematic diagram showing one example of the image ofthe bacterial colony in the petri dish.

FIG. 20A is a plan schematic diagram showing one example of the image ofthe bacterial colony in the petri dish.

FIG. 20B is a partially enlarged plan schematic diagram of neighboringregions of a marker of FIG. 3-A.

FIG. 21 is a cross-sectional schematic diagram showing a petri dish, arotational stage, and a photo sensor.

FIG. 22 is a schematic diagram showing another example of a device forharvesting a bacterial colony according to the present invention.

FIG. 23 is an operational principle of a device for harvesting abacterial colony showing an embodiment based on the present invention.

FIG. 24 is an embodiment of an imaging unit based on the presentinvention.

FIG. 25 is a relationship of the volume and the turbidity of thebacterial colony.

FIG. 26 is an example of the quantity of harvested bacteria required toacquire predetermined turbidity.

FIG. 27 is a display example of a monitor.

FIG. 28 is another display example of the monitor.

DESCRIPTION OF EMBODIMENTS

Hereinafter, embodiments of the present invention will be described withreference to the accompanying drawings.

First Embodiment

An overall configuration of an automatic harvesting device 100 will beshown in FIG. 1. Reference numeral 101 represents a low-angleillumination unit, reference numeral 102 represents a high-angleillumination unit, reference numeral 103 represents a camera, andreference numeral 104 represents a transmitted illumination unit. It ispreferable that the camera of reference numeral 103 is capable ofacquiring color information. Reference numeral 105 represents a lightshielding plate, reference numeral 106 represents a base, and referencenumeral 107 represents a petri dish. A medium 150 of bacteria isprovided within the petri dish of reference numeral 107 and a bacterialcolony acquired from a specimen to be subjected to a test is culturedthereon. As the bacterial colony, for example, a bacterial colony isused, which is cultured within a constant-temperature bath forapproximately 24 H. Reference numerals 101, 102, and 104 are connectedto a light control unit 115, light-on and light-off of referencenumerals 101, 102, and 104 may be controlled, and reference numerals101, 102, and 104 may be lit on through a predetermined combinationthereof.

As the light shielding plate 105, for example, a movable diaphragm or aliquid crystal shutter is used. The surface of the movable diaphragm issubjected to opaque black painting and liquid crystals are subjected toanti-reflection treatment. When the transmitted illumination unit 104 islit on, illumination light is set to illuminate the petri dish 107 andfurther, when the low-angle illumination unit 101 and the high-angleillumination unit 102 are lit on, a bottom surface of the petri dish 107is darkened. In general, characters may be printed on the bottom surfaceof the petri dish 107. This has an effect to prevent the characters frombeing apparent when the illumination of the low-angle illumination unit101 or the high-angle illumination unit 102 is turned on.

Reference numeral 103 represents imaging means and an imaged image istransmitted to an image input means 117. Reference numeral 119represents image processing means, and extracts an area of the bacterialcolony by processing image data transmitted to the image input means 117and further, calculates an image feature amount from each extractedbacterial colony by using image processing. The image feature amountincludes, for example, a boundary length, an area, color information,luminosity information, and a difference from background luminosity ofthe bacterial colony area. Reference numeral 118 represents groupingmeans and inputs the image inputted by using the image input means 117,a bacterial colony area 119, and the image feature amount. Referencenumeral 125 represents a secondary storage device and may store theimaged image, the processed image, data, and the position and the areaof the bacterial colony. Further, reference numeral 125 may conserve theimage feature amount.

The grouping means 118 performs grouping based on distribution offeature amounts stored every correspondence between the medium 150 thathas already acquired and a group of bacteria having a similarappearance, which is stored in the secondary storage device 125. Abacterial colony to be harvested or a candidate thereof is automaticallydetermined based on the group acquired herein. The grouping result isdisplayed on a GUI 124 of an input/output terminal 130 and a tester mayverify the grouping result and the candidate to be harvested.

When the grouping result is different from an intention of the tester,the tester registers different grouping in the grouping means 118through the GUI 124. For example, several bacterial colonies to beharvested are designated from the image, an image feature amount of abacterial colony close to the designated bacterial colonies is analyzed,and bacterial colonies having an image feature amount similar to theimage feature amount are grouped as a bacterial colony species to beharvested. The grouping means 118 extracts the candidate of thebacterial colony to be harvested and displays the extracted candidate tothe tester on the GUI 124, based on a rule which the tester designatesthrough the GUI from the grouping result. The grouping rule includes,for example, the following:

1) A group closest to the center of the feature amount among the groups,

2) A group having the size close to a predetermined size, and

3) A group close to a feature amount of a bacterial colony designated bythe tester.

In addition to the verification mode using the GUI, an automaticoperation mode is also provided and in the case of the automaticoperation mode, a bacterial colony selected by a system is justharvested without the tester's verification.

Reference numeral 108 represents transportation means of the petri dishand is used for the purpose of storing the imaged petri dish 107 in astacker 109. The petri dish may be transported by the petri dishtransportation means 108 after grouping using the grouping means 118 orthe image processing using the image processing means 119 or before theimage processing or the grouping. When a verification operation of thegrouping result by the tester is required, a comparatively long time isfrequently required for the operation. Therefore, in order to increasethroughput of the device, when imaging of the image is completed, animage of a subsequent petri dish 107 is inputted without waiting for thetester's verification operation and the tester preferably verifies andmodifies the grouping result and determine the bacterial colony to beharvested by using the image which has been already acquired. Referencenumeral 110 also represents the transportation means of the petri dishand is used for the purpose of transporting the petri dish from thestacker to a harvesting area.

Reference numeral 111 represents a harvesting needle, reference numeral112 represents a Z stage, and reference numeral 113 is an XY stage andthe petri dish transportation means 108 and 110, the Z stage 112, andthe XY stage 113 are controlled by a stage control unit 116. A petridish 114 transported from the stacker is illuminated by a transmittedillumination unit 120 and an upper illumination unit 121, the image maybe imaged by using a camera 122, and the imaged image may be inputted byusing the image input means 117.

The image inputted into the image input means 117 is compared with theimage acquired by using the camera 103 in the image processing means 119and a position deviation when the same petri dish is transported throughthe stacker is corrected. For example, the image imaged by using thecamera 103 is represented by I0 and the image imaged by the camera 122is represented by I1. The image I0 and the image I1 are imaged so thatillumination conditions most coincide with each other in two opticalsystems. By setting a background of the petri dish 107 to be black, thepetri dish 107 is imaged brightly, and as a result, a petri dish areamay be extracted by binarization processing. The petri dish area isacquired by each of the image I0 and the image I1 and substantiallymatched with a rotational center of the petri dish 107. In order toprevent an influence of a flash of the petri dish 107, luminosities atan edge of the petri dish 107 and at the outside thereof are set to beblack in terms of image processing. Thereafter, a spectrum image F(I0)by using 2D Fourier transform of the image I0 and 2D Fourier transformF(I1) of the image I1 are acquired. Since the Fourier-transform spectrumimage is not changed with respect to the position deviation of theimage, a difference between two images is only a rotation-directiondeviation of the petri dish.

In order to reduce an influence of an unevenness of brightness caused bya difference between the imaged images of the camera 103 and the camera122, when log transform is performed with respect to each of F(I0) andF(I1) and thereafter, a DC element of Fourier transform is set as anoriginal point, a distance from the original point is set as r and arotational element around the original point is set as θ, and localimage transform of r-θ is performed. The images are set as an image I2and an image I3. The images of the image I2 and the image I3 areFourier-transformed only in a θ direction of local coordinate transform.A product of the Fourier-transformed images is acquired by a correlationtheorem and inversely Fourier-transformed to acquire inter-correlationcoefficients of respective r positions. By adding up the respectiveinter-correlation coefficients, a 1D inter-correlation coefficient for θof the entire image is acquired and a peak thereof is calculated as arotational deviation amount.

Subsequently, the I0 image is rotation-transformed as large as an angleof the rotational deviation and an XY-direction deviation amount of theI0 image and the I1 image is calculated by normalized inter-correlation.A center coordinate in rotation-transforming is (cx, cy). Further, adeviation amount is (Δx, Δy, θ0). When the position of the bacterialcolony imaged by the camera of reference numeral 103 is (x, y), theposition (x′, y′) of the bacterial colony in the image of referencenumeral 122 is represented by the following equation.

x′=(x−cx)cos Δθ−(y−cy)sin Δθ+Δx

y′=(x−cx)sin Δθ+(y−cy)cos Δθ+Δy  (Eq. 1)

Further, the coordinate matching method using the 2D Fourier transformis one embodiment and besides, other methods including a method usingnormalized correlation calculation may be adopted.

Reference numeral 123 represents a micro plate and a plurality of wellsare placed. In the respective wells, for example, Ca ions of 50 mg/l, Caions of 25 mg/l, and a drug for evaluating tolerance to bacteria areadded to a Muller Hinton culture liquid so as to have differentconcentrations in the respective wells. Other liquid media may be usedand further, solid media may be used. By controlling the Z stage 112 andthe XY stage 113, the harvesting needle 111 is moved to a coordinate ofthe bacteria to be harvested of the petri dish 114 to pick up thebacterial colony. In order to pick up the bacterial colony, a forcesensor (not shown) is preferably mounted at a mounting position of theharvesting needle 111 to allow a point where the harvesting needle 111contacts the solid medium to be sensed. Thereafter, the bacterial colonyis put into a vessel 126 into which normal saline or a liquid medium isput. This operation is performed with respect to several bacterialcolonies and after the concentration of the bacterial colony in thevessel 126 becomes sufficient, a suspension in the vessel 126 is putinto any one well on the micro plate 123.

The harvesting needle 111 is changed to another one every time differentbacterial colony species are harvested up. There is a case in which asufficient number of bacterial colonies cannot be cultured within onepetri dish. In this case, there is a case in which bacteria cannot becultured in all of set wells on the micro plate 123, in one petri dish.Therefore, in this case, bacteria of the specimen are cultured in theplurality of petri dishes, a particular bacteria species is determinedfrom the plurality of petri dishes, and the particular bacteria in theplurality of petri dishes are moved to one micro plate 123 from theplurality of petri dishes. As a result, the MIC method may be executedeven at a comparatively short culture time. As described above, when thebacterial colony to be harvested is selected from the plurality of petridish images, the plurality of petri dish images of the same specimenneed to be easily displayed in the GUI 124 of the input/output means130. The entirety of the automatic harvesting device 100 is controlledby an entire control unit 140.

FIG. 2 shows a detailed configuration of illumination means such as thelow-angle illumination unit 101 and the high-angle illumination unit102. The low-angle illumination unit 101 is constituted by divisionlight sources 2011, 2012, 2013, and 2014 and light-on thereof may beindividually controlled by using the light control unit 115. Thehigh-angle illumination unit 102 is constituted by division lightsources 2021, 2022, 2023, and 2024 and light-on thereof may also beindividually controlled by using the light control unit 115. Since thesurface of the bacterial colony is comparatively smooth, a directlyreflected place is imaged brightly by the illumination of the surface.

As shown in FIGS. 2A and 2B, illumination is performed for eachdirection, such that luminosity on a plane including illumination, thecenter of the bacterial colony, and an optical axis of the camera may beacquired. When acquiring a detailed shape of the bacterial colony, totaleight-time imaging of four-time imaging of the image illuminated by thedivision light sources 2011, 2012, 2103, and 2014 one by one andfour-time imaging of the image illuminated by the division light sources2021, 2022, 2023, and 2024 one by one by the control of the lightcontrol unit 115 is preferable. However, in order to shorten an imagingtime, it is preferable that several images are illuminated and imaged atthe same time. Meanwhile, in order to shorten the imaging time, forexample, total two images of one image illuminating the division lightsources 2011, 2012, 2013, and 2014 simultaneously and one imageilluminating the division light sources 2021, 2022, 2023, and 2024simultaneously will be enough.

When directly reflected light is detected, color information of abacterial colony therearound is lost. In order to discriminate thebacterial colony for each kind, the color information is important.Therefore, we selected an angle at which the directly reflected light isnot detected as much as possible in low-angle illumination. Theillumination of the low-angle illumination unit 101 is performed at thehighest angle, θ at the end of the petri dish and according to anevaluation result, it could be seen that when 0 is 45° or lower, thecolor information may be acquired without an influence of the directlyreflected light in most bacterial colonies.

Even in high-angle illumination, illumination is performed at thehighest angle, θ₂ at the end of the petri dish and it could be seen thatwhen the angle is over 80°, the medium 150 is inclined by surfacetension at the end of the petri dish according to the medium 150 and themedium itself is brightly detected by direct reflection. Therefore, eventhough the high-angle illumination is the highest-angle illumination,the illumination angle is set not to be over 80°.

Since the directly reflected light is not detected in spite of groundillumination from a low angle in most cases in the low-angleillumination formed by the division light sources 2011, 2012, 2013, and2014 by setting the illumination angle as above, acquired informationamount is not large in spite of one-by-one illumination. As a result,the low-angle illumination is preferable in that simultaneousillumination and imaging in all directions shortens the imaging time. InFIG. 3, substantially the same configuration as FIG. 2 is configured byline-shape illumination. Illumination of a low-angle illumination unit301 is constituted by division light sources 3011, 3012, 3013, and 3014and illumination of a high-angle illumination unit 302 is constituted bydivision light sources 3021, 3022, 3023, and 3024. Light-on of therespective light sources may be individually controlled by using thelight control unit 115.

In the configuration of FIG. 2, in order to further increase angledetection resolution, a middle-angle illumination unit 420 may beinserted between a low-angle illumination unit 401 and a high-angleillumination unit 402 as shown in FIG. 4. In this case, illumination ofthe low-angle illumination unit 401 is divided into four division lightsources 4011, 4012, 4013, and 4014, illumination of the high-angleillumination unit 402 is divided into four division light sources 4021,4022, 4023, and 4024, and illumination of the middle-angle illuminationunit 420 is divided into four division light sources 4201, 4202, 4203,and 4204 and light-on thereof may also be individually controlled byusing the light control unit 115.

When it is considered that classification is performed based on thedirectly reflected light of the bacterial colony, the directly reflectedlight from substantially the same normal direction of the bacterialcolony needs to be detected even at any position of the petri dish.

However, the petri dish is, in general, φ90 mm and in order to achieveit, the high-angle illumination unit 102, 302, or 402 and the camera 103need to be significantly separated from the petri dish 107. In order toreduce the separation gap as small as possible, the camera may adopt atelecentric optical system. By this method, a distance between a lens ofan imaging system and the petri dish may be shortened.

Further, in regard to an illumination system, the high-angleillumination unit 102, 302, or 402 is constituted by a set of aplurality of LEDs, a micro lens is installed in each LED, and a methodusing comparatively parallel light is adopted, thereby shortening adistance between the positions of the illumination and the petri dish107. Further, as another configuration, a system shown in FIG. 17 isalso used. Reference numerals 1701 and 1703 represent light sources,reference numerals 1702 and 1704 represent parabolic mirrors, and thelight sources 1701 and 1703 are placed at focus positions of theparabolic mirrors 1702 and 1704, respectively. By this configuration,light emitted from the light sources 1701 and 1703 to the parabolicmirrors 1702 and 1704, respectively, is reflected on the parabolicmirrors 1702 and 1704 to be parallel light, thereby radiating the petridish 107. As a result, light may be illuminated from substantially thesame direction at a random position on the petri dish 107. Referencenumerals 1702 and 1704 may be spherical mirrors instead of the parabolicmirrors, respectively.

FIG. 5A shows an actualized state of a feature of a bacterial colony byeach illumination. In the high-angle illumination, according to theposition of reflection light, an angle of the bacterial colony at theposition of the reflection light may be acquired. The normal-linedirection of the bacterial colony at the position from which thedirectly reflected light is detected is expressed by (Vc+Vi)/2 by usinga unit vector Vc from the position of the reflection light toward thecamera lens and a unit vector Vi from the position of the reflectionlight to the illumination, as shown in FIG. 5B. It may be estimatedthat, when the bacterial colony is flat, a position where the reflectedlight is detected is around the center of the bacterial colony and whenthe position of the reflected light is around the bacterial colony, thebacterial colony has a height from the medium. Further, when thebacterial colony has a shape collapsed from a dome shape, the reflectedlight is detected at a plurality of positions. On the other hand, in thelow-angle illumination, even when a surface shape of the bacterialcolony is slightly changed, luminosity is not changed, but color andluminosity information of the surface of the bacterial colony is easilyacquired.

FIG. 6 shows the distribution of the bacterial colonies in the petridish. Reference numeral 601 represents the petri dish and as shown byreference numeral 602 thereon, there is a slightly bright area near theoutside of the petri dish, which is generated as the medium 150 isinclined due to diffused reflection from the petri dish or an influenceof surface tension at an edge of the petri dish. An area where theluminosity or color of the medium 150 is changed by a secretory fluid ofthe bacterial colony is shown by reference numeral 603. Referencenumeral 604 represents a small-sized bacterial colony havingcomparatively low contrast from the medium 150 and reference numeral 605represents a bacterial colony having high contrast.

An image signal acquired by imaging the petri dish 107 (601) with theimaging means 103 is received by the image processing means 119, and thebacterial colony having high contrast is easily detected. But, thebacterial colony at reference numeral 603 has low contrast and referencenumerals 602 and 603 are spaced apart from each other due to a changefrom average luminosity or a color of the medium 150. When the bacterialcolony is detected through a difference in color or luminosity from theaverage color or luminosity of the medium 150 by the image processingmeans 119, the area which is not the bacterial colony, such as referencenumeral 602 or 603 is also extracted as the bacterial colony under acondition to extract 604. Therefore, in the embodiment, an isolatedbacterial colony may be extracted by using three steps below.

(1) removing a low-frequency component which is a feature of referencenumeral 602 or 603 by means of a spatial bandpass filter.

(2) acquiring luminosity or color of an area considered as the medium150 based on local luminosity around a focused area and comparing theacquired luminosity or color with the local luminosity or color andextracting the focused area as a candidate of the bacterial colony whenthe focused area is different from the area of the medium in theluminosity or color.

(3) evaluating the distribution of the luminosity or the color of thearea extracted as the candidate of the bacterial colony and judging acase in which the luminosity distribution is similar to the dome shapeas the bacterial colony.

FIG. 7 shows a way of thinking of an algorithm in step (2). FIG. 7 showsthe distribution of luminosity in a 1D direction where an imaged imageis present. Reference numerals 701 and 702 represent bacterial coloniesto be extracted, but when a simple binarization method is adopted, anarea of reference numeral 703 is extracted as the bacterial colony.Therefore, when it is judged whether the area of reference numeral 701is the bacterial colony, the judgment may be performed based on the areaof reference numeral 703 therearound and when it is judged whether thearea of reference numeral 702 is the bacterial colony, the judgment maybe performed based on an area of reference numeral 704.

As the method of (3), for example, quadratic function fitting may beused. In the processing of (2), even though an area having highluminosity is extracted based on the neighborhood, an area where theplurality of bacterial colonies are overlapped with each other may beextracted as shown by reference numeral 606. In an aggregate of theplurality of bacterial colonies, since the aggregate is not configuredby only a single bacterium in some cases, the aggregate should beremoved. When transmitted image luminosity of the candidate area of thebacterial colony acquired by (2) is I(x, y), for example, an equationbelow is calculated in the candidate area of the bacterial colony,coefficients A, B, C, D, E, F, G, and H are calculated by aleast-squares method so that Error has a minimum value, and a methodusing the bacterial colony as the isolated bacterial colony may beadopted only if Error/S is equal to or less than a threshold when anarea of the candidate area of the bacterial colony is set as S. Further,the quadratic function fitting may be calculated similarly by Gaussianfunction approximation.

Error=Σ(Ax ² +Bx+C+Dy ² +Ey+F+Gxy+H−I(x,y))²  (Eq. 2)

In the step described by (2), the image of the medium 150 is not steeplychanged, but it was assumed that luminosity in the image is steeplychanged by only the bacterial colony. However, actually, other changesin luminosity also occur. The biggest problem is a character written onthe petri dish. Reference numeral 801 of FIG. 8A(a) represents an imageimaged by illuminating the petri dish 107 from the bottom as shown inFIG. 8A(b) with only the transmitted illumination unit 104 shown in FIG.1, an image 802 of the petri dish 107 of FIG. 8B(a) is an image imagedby illuminating the petri dish 107 at the low angle with the low-angleillumination unit 101, 301, or 401 as shown in FIG. 8B(b), and an image803 of the petri dish 107 of FIG. 8C(a) is an image acquired byilluminating the petri dish 107 from the top with the high-angleillumination unit 102, 302, or 402 as shown n FIG. 8C(b). The characteris the clearest in the image 801 of the petri dish 107 in FIG. 8A(a),but it can be seen that in general, a printed character 805 for checkingtraceability is implemented on the bottom of the petri dish, in thepetri dish 107 used to culture bacteria. In a hospital or an inspectioninstitute, a specimen ID may be written with magic ink in order todistinguish a plurality of similar bacteria culturing petri dishes. Assuch, there is a case in which the character on the bottom of the petridish may be clearer than the bacterial colony.

In the image illuminated at the low angle, such as the image 802 of thepetri dish 107 of FIG. 8B(a), the printed character 805 is not almostviewed, but may be viewed in the medium 150 (corresponding to a medium610 of FIG. 6) having very high transparency. The character is lessclear in the image 803 of the petri dish 107 in FIG. 8C(a) than in theimage 801 of FIG. 8A(a), but the character may be viewed more clearlythan in the image 802 of the petri dish 107 in FIG. 8B(a). Referencenumeral 804 represents the bacterial colony and reference numeral 805represents the printed character.

When the petri dish is illuminated from the bottom as in the image 801of FIG. 8A(a), printing of the printed character 805 blocks light, andas a result, the petri dish is imaged very excellently. In the case ofthe low-angle illumination of the image 802 of the petri dish 107 inFIG. 8B(a), most of the light is reflected on the surface and does nottransmit the medium 150, and as a result, the printed character 805 isbarely viewed. Since the image 803 of the petri dish 107 in FIG. 8C(a)is illuminated from the top, light is inputted into the medium 150rather than at the low-angle illumination and the printed character 805is clearly viewed. In a case other than the transmitted illuminationusing the transmitted illumination unit 104, the light shielding plate105 is provided, such that the character of the image 803 of the petridish 107 in FIG. 8C(a) is difficult to be seen, but it is hard to becompletely invisible. Therefore, in step (2), first, it is necessary tospecify printing on the bottom of the petri dish such as the printedcharacter 805 or a recording area such as the character, a label, andthe like which becomes obstacle when the luminosity and the color of themedium 150 is recognized by local area processing.

An algorithm of extracting the candidate area of the bacterial colony byspecifying and excluding the recording area will be described withreference to FIG. 9. First, a dark area A is extracted from thetransmitted illumination image acquired by imaging with the imagingmeans 103 a transmitted illumination state of the petri dish 107 fromthe bottom by using the transmitted illumination unit 104 (S901). Thedark area A is generally the area of the bacterial colony or therecording area of the bottom of the petri dish. Subsequently, in an areaother than the area A, the petri dish 107 is illuminated from the top byusing any one of the high-angle illumination unit 102, 302, or 402 andthe low-angle illumination unit 101, 301, or 401 or combinations of theplurality of units, and the average luminosity of the medium 150 isacquired from the upward illumination image acquired by using the camera103 (S902). Herein, the image by the high-angle illumination using thehigh-angle illumination unit 102, 302, or 402, the image by thelow-angle illumination using the low-angle illumination unit 101, 301,or 401, or both sides may be used. The reason is that there is a colonywhich is apt to be actualized in the image by the high-angleillumination and there is another colony which is apt to be actualizedin the image by the low-angle illumination.

The most robust algorithm preferably performs both sides. Furthermore,when illumination is performed for each illumination direction by usingthe low-angle illumination unit 101, 301, or 401 or the high-angleillumination unit 102, 302, or 402, calculating averaged syntheticimages of images sequentially illuminated from all directions,respectively may detect a bacterial colony which does not depend on theposition of the petri dish.

In general, even when the medium 150 is discolored by a secretory fluidof the bacteria, the amount of discolor of the medium 150 is not solarge. Therefore, subsequently, luminosity and color of the local medium150 are acquired from an area having brightness and color close to theaverage luminosity (S903). An area having luminosity and color which arelargely different from the local luminosity and color is extracted asthe candidate area of the bacterial colony (S904).

Subsequently, a step of extracting an image feature amount of thebacterial colony from the extracted candidate area of the bacterialcolony will be described with reference to FIG. 10. First, a bacterialcolony 1005 is extracted from the candidate area of the bacterial colonyextracted in S904 of FIG. 9 by using the method described in (3)(S1011). The image feature amount of the bacterial colony for groupingthe bacterial colonies is calculated from the area of each extractedbacterial colony (S1012).

As the image feature amount, at least feature amounts described beloware calculated by processing a high-angle illumination image 1001(illuminated by the high-angle illumination unit 102), a low-angleillumination image 1002 (illuminated by the low-angle illumination unit101), and a transmitted illumination image 1003 (illuminated by thetransmitted illumination unit 104). Further, a plurality of high-angleillumination images are sequentially acquired and individually processedby individually illuminating the high-angle illumination image 1001 fromthe plurality of directions to acquire a feature which has excellentdistinctiveness.

(a) Surface irregularity: 1007

Since a normal direction of a colony at a position where the surfaceirregularity is observed can be calculated according to the position ofthe directly reflected light as shown in FIG. 5, the surfaceirregularity is acquired therefrom.

(b) Texture: 1007

The texture is also acquired from a state in which the directlyreflected light is emitted. When the surface becomes rough, the directlyreflected light of a small area is observed as a non-point target withrespect to the center of the bacterial colony for each direction, and asa result, the texture is used.

(c) Color: 1008

The color feature is calculated from the image acquired by the low angleillumination or the image acquired by the transmitted illumination. Thecolor feature may be auxiliary calculated from the image illuminated atthe high angle. As a color space, a Lu*v* space or an HSV space may beused as well as general RGB. In particular, when an RGB image is used inthe case where the transmitted illumination image is used, luminosity ischanged according to a height of the bacterial colony from the medium150, and as a result, there is a strong possibility that the featureamounts will not be statistically independent from each other when aheight feature and a color feature of the bacterial colony areseparately set. As a result, the Lu*v* space or HSV space in which theluminosity and the color are independent has a large merit.

(d) Shape (size, circularity, long axis/short axis): 1009

The size of the area, the circularity thereof, or a long axis/short axiswhen a shape is elliptically approximated is acquired from the imagewith respect to the bacterial colony 1006 selected among the detectedbacterial colonies 1005. The circularity is acquired by dividing acircumferential length of the area by the square root of an area. By thecircularity, an edge of bacteria having an amoeboid form is actualized.

(e) Luminosity, color, transmittance, and cubic shape: 1010

The luminosity, color, transmittance, and cubic shape are acquired fromthe transmitted illumination image 1003. The transmittance and theheight from the medium 150 can be acquired based on the luminosity anddirect reflection of the high-angle illumination image. FIG. 11 showsthe relationship between the high-angle illumination image and thetransmitted illumination image. Reference numeral 1101 represents aluminosity level at each position of the high-angle illumination imageilluminated from any one direction and reference numeral 1102 representsa luminosity level at each position of the transmitted illuminationimage.

The transmitted illumination may be used only when the medium 150 hashigh light transmittance and the transmitted illumination becomes darkat the position of the bacterial colony, and as a result, the thicknessof the bacterial colony increases and further, becomes dark. When anyluminosity in the bacterial colony is set as IC(x, y) and an averagevalue of the luminosity of the medium 150 is set as IM, the thicknessD(x, y) of the bacterial colony at a position of (x, y) is acquired byan equation below.

D(x,y)=−G(log IC(x,y)−log IM)  (Eq. 3)

Herein, G represents a gain determined for each bacterial colonyspecies.

It is assumed that the reflected light by the high-angle illuminationcan be detected at a position (X, Y). In this case, a normal vector isinclined from a vertical direction by θ. In this case, a change inthickness when the position is deviated by Δx becomes −Δx tan θ. Herein,when a difference of D(x, y) is calculated as below.

D(X+Δx,Y)−D(X,Y)=−G(log IC(X+Δx,Y)−log IC(X,Y))=−Δx tan θ

That is,

G=Δx tan θ/(log IC(X+Δx,Y)−log IC(X,Y))  (Eq. 4)

The difference is acquired by Eq. 4, and as a result, the thickness ofthe bacterial colony is acquired at any position. Therefore, when only aplace where the directly reflected light is generated is discovered inthe medium 150 having comparatively high transmittance, the placebecomes a very important feature amount in identifying the bacterialcolony in which the cubic shape and volume are acquired. For example,the maximum height, volume, average height, and outer peripheral heightof the bacterial colony are calculated from the information. A vectorusing the features as components is acquired and the vector is set asthe image feature amount of each bacterial colony.

The bacterial colonies are classified into the bacterial colony for eachbacterial species in the grouping means 118 by using information of theimage feature amount for each bacterial colony acquired by the imageprocessing means 119. The method of classifying the bacterial coloniesinto the bacterial colony for each bacterial species basically includestwo methods. One is a method of classifying the bacterial colonies foreach bacterial species which is already registered and the other is amethod of classifying the bacterial colonies for each kind of bacterialcolony only by the bacterial colony image of the petri dish which isgiven.

The first method will be described with reference to FIG. 12. Referencenumeral 1201 of (a) represents a feature amount database (DB) and thedatabase may search data by using an index 1201 shown in (b). Abacterial species 12021 is not particularly limited to one bacterialspecies and may be a group consisting of several bacterial species, buteven in this case, the bacterial species 12021 is a group consisting ofbacterial species having similar appearances. Next, a medium 12022 willbe described. The bacterial colony may be limitatively generated by themedium and further, in some media, the color of the bacterial colony islargely changed according to the bacterial species. As a result,information of the medium is very important in classifying bacteria. Inthe case of a specimen 12023, bacteria which are easily generated arechanged depending on whether the specimen acquiring bacteria is, forexample, patient's blood, feces, or urine. The information is alsoimportant in classifying bacteria in order not to overlook importantbacteria. Further, a culturing condition 12024 of the bacterial species12021 is important in classifying the bacterial species. Last, featureamount data 12025 will be described.

The feature amount data 12025 is vector data and is constituted by a setof vector data of a plurality of same kind of bacterial colonies. Forexample, reference numerals 1203, 1204, and 1205 shown in FIG. (c)represent vector groups of bacterial colonies of different bacterialspecies imaged with the same medium, specimen, and culturing condition,respectively. Now, when a feature amount vector acquired from abacterial colony of a new unknown bacterial species is 1206, the mostsimilar feature amount vector 1205 is adjacent to 1206 in the presentinvention, and as a result, 1206 is also judged as the same bacterialcolony as 1205.

A typical bacterial colony is registered in the database as describedabove. But there is a case where bacteria cultured from the specimen maynot be particularly registered in the database. In order to deal withthis case, a method of interactively grouping the feature amount vectoracquired from the petri dish by image processing without using thedatabase may be used.

This algorithm will be described with reference to FIGS. 13A and 13B. Ina feature amount space 1301 of FIG. 13A, reference numerals 1303, 1304,and 1305 represent feature amount vectors of the bacterial colonieswhich correspond to a bacterial colony 1306, a bacterial colony 1307,and a bacterial colony 1308, respectively in an image 1302 of the petridish shown in FIG. 13B. When a tester instructs that typical examples ofbacterial colonies to be grouped are 1303, 1304, and 1305 to the systemwhile a drawing of the feature amount space shown in FIG. 13A isdisplayed on the GUI 124 described in the configuration of the entiredevice of FIG. 1, the grouping means 118 groups the feature amountvectors by using a clustering method which is generally known.

As the clustering method, a method such as, for example, an EMalgorithm, k-means, or purge k-means may be used and an instructionresult of the tester may be used as an initial grouping state beforeexecuting this algorithm. Further, not the clustering algorithm but ageneral classification algorithm, for example, a nearest neighbor methodor a Naive Bayes method may be used. When the classification algorithmis used, the classification algorithm is executed with a class in whichsamples are instructed one by one.

When the feature amount is largely distant from a feature of a typicalbacterial colony instructed by the tester, the grouping means 118excludes the classification result from grouping as the unknownbacterial colony. Further, bacterial colonies which are equally spacedapart from the feature amount of the plurality of bacterial coloniesinstructed as the typical bacterial colonies are excluded from groupingas bacterial colonies of which a grouping destination is unclear. Thisis inevitable in order to prevent tolerance of the drug from beingevaluated by using bacteria which is not presented originally, inimplementing, for example, the MIC method. Further, the to systemselects a sample to be harvested based on the grouping result. When aplurality of bacterial colonies are cultured, the tester preferablyselects the bacterial colony to harvest a bacterial colony close to thebacterial colony selected as the typical bacterial colony in terms ofthe feature amount in the grouping result.

The result processed by the grouping means 118 is displayed on the GUIscreen 124 of the input/output terminal 130. The tester verifies thegrouping result shown in the GUI 124 and if the grouping result is apreferable classification result or a harvesting result, the testerinstructs the system to harvest the bacterial colony according to thisresult. When the grouping result is insufficient, a bacterial species ofa specific bacterial colony is instructed and newly inserted into thesystem through the GUI. The grouping means 118 adds the bacterialspecies as new instruction data of, for example, the Naive Bayes methodand classifies the added data and shows the grouping result to thetester almost in real time. Besides, the data may be treated as newinstruction data of K-NN classification or a group of added instructiondata of the tester may be reflected to an initial state of K-meansclustering. Further, other known classification algorithm or clusteringalgorithm may be used and the classification result after adding theinstruction data is displayed to the tester to allow the tester toimmediately judge whether the classification result is a satisfactorygrouping result.

A bacterial colony in which the tester cannot specify the bacterialspecies is also present. In this case, the tester registers thebacterial species as the bacterial colony in which the bacterial speciescannot be specified in the grouping means 118 through the input/outputterminal 130. The grouping means 118 specifies, for example, two groupswhich are the closest in a feature space of the bacterial colony inwhich the bacterial species cannot be specified and boundary surfaces ofthe two groups are moved to the centers of the groups, respectively, inorder not to group the bacterial colony in which the tester cannotspecify the bacterial species.

Further, the grouping result is not limited to be performed by only onepetri dish. The reason is that a plurality of wells are placed on themicro plate, all wells may not be filled with bacteria by only one petridish, and bacteria of one specimen are cultured by a plurality of petridishes. In this case, the images of the plurality of petri dishes may besimultaneously displayed to allow the tester to verify the images of theplurality of petri dishes and select an instruction sample.Alternatively, one of them is enlarged and displayed on the GUI 124 andthe image may be easily switched by operating a mouse, a track ball, ora keyboard of the input/output terminal 130.

In the system of the present invention, imaging is performed underdifferent optical conditions. When the tester does not verify the imageunder a plurality of optical conditions, the tester may not specify anaccurate bacterial species. Therefore, the images under the plurality ofoptical conditions may be displayed on the GUI 124. The GUI 124 maydisplay an image under a predetermined optical condition on the GUI 124by operating the mouse, track ball, or keyboard of the input/outputterminal 130 or the GUI 124 may display the images under the pluralityof optical conditions simultaneously while displaying only some of theimages of the respective petri dishes.

As described above, this system performs grouping based on the imagefeature amount, shows the grouping result to the tester through the GUI124, and allows the tester to modify the grouping result, thereby makingit possible to select the same bacteria colony and apply the MIC method,but an inexperienced tester may not perform accurate grouping by only anappearance of the bacterial colony. Therefore, more detailed analysis isperformed except for application of the MIC method. Infraredspectroscopy or Raman spectroscopy is performed as the analysis method.As the bacterial colony to be harvested, bacterial colonies at thecenter of a feature amount of each grouped bacterial colony and at aboundary of each group are preferably selected.

In the Raman spectroscopy or infrared spectroscopy, it takes a long timeto image an image. In particular, in the infrared spectroscopy, sincemoisture is not generally permeated by light, an analysis target needsto be dried. Therefore, the bacterial colony is put into the base inwhich the harvested bacterial colony is once analyzed by using 111 and afunction to identify one kind of the bacterial colony is provided.

The case using the infrared spectroscopy will be shown in FIG. 14.Reference numeral 1401 represents an infrared light source, referencenumeral 1402 represents a half mirror, reference numeral 1403 representsa mirror, and reference numeral 1404 represents a stage and may changethe position of a mirror 1405. The stage 1404 is connected with a stagecontroller 116 and the position of stage 1404 may be controlled by thesystem. Light reflected on the mirror 1403 interferes with reflectedlight of the mirror 1405 at the position of the half mirror 1402 and theintensity of the light is changed by a wavelength. Reference numeral1407 represents a base. Reference numeral 1408 represents a harvestedbacterial colony and the light interfered by the half mirror 1402 isradiated to a bacterial colony 1408 after a polarization directionthereof is adjusted to a polarization filter 1406.

A polarization direction of light that transmits the polarization filter1406 generally becomes P polarization and an illumination angle is setto be approximate to a Brewster angle to control direct reflection fromthe bacterial colony 1408 to be reduced. Further, even though the angleis not the Brewster angle, polarization directions of a polarizationfilter 1410 at a detection side and the polarization filter 1406 at anillumination side are perpendicular to each other to suppress thedirectly reflected light. In the case of a polarization direction oflight reflected by the bacterial colony 1408, since a condition tocapture most light diffused inside the bacterial colony is changed by acomposition of the bacterial colony 1408, the polarization filter 1410may rotate to adjust a polarization characteristic. Reference numeral1409 represents a condensing lens. In order to perform only analysis ofnear-infrared light, a general lens made of quartz may be used and areflection object lens may be used in order to analyze mid-infraredlight.

Reference numeral 1411 represents light detection means. As the lightdetection means 1411, a photo multiplier tube detecting weak light at ahigh gain is appropriate and preferably, a photo multiplier tube of amulti anode in which the photo multiplier tube is placed in a 2D shapeis appropriate to determine a spatial distribution. Besides, a method ofplacing an image intensifier on a front surface of a detector anddetecting the weak light by using a 2D CCD or CMOS sensor at a rear endmay be used and a general CCD sensor or CMOS sensor may be placed.

A detected optical signal is expressed by an equation below.

F(X)=∫B(λ)cos(2πλX)dλ  (Eq. 5)

Herein, B(λ) is the detected amount of light for each wavelength λ oflight. X represents an optical path difference between a path reflectedon 1403 and a path reflected on 1405, which is generated by controllingthe stage 1404. Detected F(X) is transmitted to 117 and B(λ) iscalculated by Fourier transform based on F(X) for each X.

That is,

B(λ)=∫F(X)cos(2πλX)dX  (Eq. 6)

When light is detected by the photo multiplier tube of a multi-anodeplaced in the 2D shape or the CCD, B(λ) is calculated at a positioncorresponding to each pixel. In general, when a reflection spectrum oflight is acquired by the infrared spectroscopy, an absorptioncoefficient of water is large, such that measurement precision issignificantly deteriorated by moisture. For enabling the measurement, asone method, the bacterial colony 1408 is formed in a dried power shape.

As another method, there is a method in which only near-infraredwavelength that less absorbs water is analyzed. When the near-infraredwavelength is analyzed, an absorption strength decreases, and as aresult, it is difficult to identify bacteria, but important informationis given to discriminate bacteria. When the near-infrared light is used,the bacterial colony 1408 need not be dried, and as a result, moisturemay be kept in the base 1407. Therefore, moisture is inputted into themicro plate 123 after measurement to perform the MIC method as it is.

When only the near-infrared light is used without using infrared light,the bacterial colony need not be dried, and as a result, the opticalsystem described in FIG. 14 may be placed directly around the petri dish114 of the optical system shown in FIG. 1 and the bacterial colony onthe petri dish may be directly analyzed.

FIG. 15 shows a correspondence of a wave number of light and lightabsorbance. As the light absorbance, the intensity of the lightdetection means 1411 may be used as it is. But, when the bacterialcolony 1408 is not uniform, it is difficult to set the detected amountof light as the light absorbance. In such case, setting a lightintensity having a wave number to acquire the maximum amount of light asa reference and a rate of intensity of light detected by each pixel ispreferably converted into the light absorbance. The correspondence ofthe wave number and the light absorbance acquired as above is comparedwith a correspondence of a wave number and light absorbance alreadyregistered in the secondary storage device 125 and bacteria of the samespecies are extracted. Further, light absorbances of a plurality ofbacterial colonies acquired in the same petri dish or the same specimenare compared with each other to discriminate the bacterial species.

According to the embodiment, by using the image acquired by illuminatingthe petri dish from the top and the image acquired by transmitting andilluminating the petri dish from the bottom, the image of the bacterialcolony may be separated from an image of a character or a symbol writtenor formed on the petri dish to be detected and the bacterial colonyspecies may be classified and accurately harvested by image processing.

By using images illuminated from a plurality of elevation angledirections, it makes possible to more accurately extract the image ofthe bacterial colony, and further, since the images may be classified byusing feature amounts of a plurality of different images in theillumination direction, the bacterial colony species can be classifiedmore accurately. As a result, the bacterial colony species may beclassified and accurately harvested automatically without human hands.

Second Embodiment

In the system configuration shown in FIG. 1, as the imaging means, thecamera capable of acquiring the 2D-shaped image is used and in thisscheme, the optical system tends to be complicated. As another scheme toprevent the complicatedness, a scheme using a detection system as alinear sensor is used. This system may use the camera 103 in theconfiguration of FIG. 1 as the linear sensor. In this case, a pulse ofthe integral multiple of a control pulse of the stage 116 is set as anexternal synchronization signal of the linear sensor, such that an imagein which a magnification is not changed with respect to a drivingdirection of the stage can be acquired. However, in order toindependently acquire the illumination images of the low-angleillumination unit 101, the high-angle illumination unit 102, and thetransmitted illumination unit 104, the illumination should be switchedand the stage should be operated in each switching, which isdisadvantageous in terms of throughput.

An embodiment for solving the problem is shown in FIG. 16. In FIG. 16,the same reference numerals refer to the same elements described inFIG. 1. Herein, in the to configuration shown in FIG. 16, linear sensors1601, 1602, and 1603 are installed instead of the camera 103 of FIG. 1.By setting external synchronization of the linear sensors as the integermultiple of the control pulse of the stage 116, the image in which themagnification is not fluctuating can be acquired. The linear sensor 1601images the transmitted illumination image by the transmittedillumination unit 104, the linear sensor 1602 images the high-angleillumination image by the high-angle illumination unit 102, and thelinear sensor 1603 images the low-angle illumination image by thelow-angle illumination unit 101.

Third Embodiment

Hereinafter, a third embodiment of the present invention will bedescribed in detail. However, the present invention is not limited tothe embodiment described herein.

(Configuration of Device for Harvesting Bacterial Colony)

FIG. 18 is a schematic diagram showing one example of a device forharvesting a bacterial colony according to a third embodiment. First, inorder to verify whether the microorganism (bacteria) is present in thespecimen collected from a patient, the specimen is applied to the agarmedium within a petri dish 1800 and inputted into an incubator toculture the microorganism (bacteria) within the petri dish. Theculturing is generally performed for 12 hours or more. When themicroorganism (bacteria) is present in the specimen, the bacterialcolony is generated in the petri dish 1800 by the microorganism afterculturing.

The petri dish 1800 in which the bacterial colony is grown is set in afirst buffer station 1801. A structure of the first buffer station 1801is not particularly limited, but a plurality of stages may be installedand the plurality of petri dishes may be stored in a vertical direction,and if a space is permitted, the petri dishes may be spread and storedin a horizontal direction. The petri dishes 1800 set in the first bufferstation 1801 are one by one transported to an imaging stage 1803 byfirst transportation means 1802. An identification code fordiscriminating each petri dish is preferably provided in each petridish. As an identifier, a barcode label may be attached to the petridish in advance and read and in the first buffer station 1801, anidentification code corresponding to the position to store each petridish 1800 may be inputted into a control unit 1802 before being set inthe device for harvesting the bacterial colony.

The petri dish 1800 transported to the imaging stage 1803 stops at aposition directly below a camera 1804 for imaging an image of the petridish 1800. In general, in order to avoid water droplets attached to theinside of a cover from dropping, the petri dishes is treated with thecover formed downward and the bottom surface formed upward. In the petridish 1800, the cover is preferably separated by reversing the petri dish1800 up and down so that the bottom surface is formed downward in themiddle of a transportation path until the petri dish 1800 stops at theposition directly below the camera 1804. The image of the petri dish1800 is acquired by using the camera 1804. Since the image can specifythe position of the bacterial colony, the image is imaged with requiredresolution (in general, approximately 0.1 mm). In the case where desiredresolution is not acquired when the image of the entire petri dish isacquired, the image may be imaged by moving the stage 1803 to which thepetri dish 1800 is fixed or the camera 1804 and increasing an imagingmagnification. One petri dish image or one petri dish image in which aplurality of images are synthesized is acquired.

The imaged image is stored in storage means (memory) within the controlunit 1820. Thereafter, bacterial colonies in the petri dish areautomatically detected by using the imaged petri dish image and abacterial colony to be harvested is selected by selection means thatselects some of the detected bacterial colonies. A coordinate of theselected bacterial colony is given by an original point and a coordinateaxis determined within the petri dish and stored in a storage unit. Theselection means and the coordinate allocation method will be describedbelow in detail. The petri dish 1800 in which storing the coordinate ofthe bacterial colony to be harvested is completed is transported to asecond buffer station 1806 by second transportation means 1805. In thesecond buffer station 1806, the plurality of stages may be installed andthe plurality of petri dishes may be stored in the vertical direction,and if the space is permitted, the petri dishes may be spread and storedin the horizontal direction, as in the first buffer station 1801.Herein, when there is a spatial limit, the first buffer station 1801 andthe second buffer station 1806 may be commonly used. In this case, forexample, the petri dishes imaged from the bottom of the petri dishesstacked the buffer station are sequentially extracted and when imagingand predetermined processing are completed, the petri dishes may bemanaged without changing the order of the stacked petri dishes bysequentially stacking the petri dishes on the top of the stacked petridish. By this configuration, one buffer station may serve as two bufferstations.

Subsequently, the petri dishes stored in the second buffer station 1806are transported to a harvesting stage 1808 by third transportation means1807. The imaging stage 1803 and the harvesting stage 1808 are installedin different areas. They may be independently operated. The harvestingstage 1808 includes image displaying means 1809, a harvesting tool 1810including a driving unit capable of moving in X, Y, and Z axisdirections, a rail 1811 of the harvesting tool, a test tube 1812, abacterial liquid concentration adjusting system 1813, and an imageoutputting device 1821. The image outputting device 1821 reads the imageof the petri dish imaged by the imaging stage, from the storage means ofthe control unit 1820. Further, the image of the petri dish is displayedby the image display means 1809. The displayed images and the imagedimages are compared by comparison means within the image outputtingdevice. The original point and the coordinate axis of the petri dish ofthe imaged image, and the coordinate of the bacterial colony to beharvested are read from the storage means and matched with the positionof the displayed image, and a target bacterial colony is harvested. Thecomparison means of the imaged image and the displayed image will bedescribed below in detail.

After harvesting, the bacterial colony is inputted into the test tube1812 and the concentration thereof is adjusted to a desiredconcentration by using the bacterial liquid concentration adjustingsystem 1813. The concentration is adjusted by increasing or decreasingthe quantity of the bacterial colonies or the amount of solvent (forexample, normal saline). The test tube 1812 where the adjustment of theconcentration is completed is mounted on a suspension rack 1815 of asuspension discharging system 1814 and appropriately carried out outsidethe device.

The imaging stage 1804, the harvesting stage 1808, the first bufferstation 1801, the second buffer station 1806, the first transportationmeans 1802, the second transportation means 1806, and the thirdtransportation means 1807 are automatically controlled by the controlunit 1820.

In FIG. 18, the control unit 1820 and the image outputting device 1821have different housings and may be integrated with each other.

(Method for Acquiring Coordinate of Bacterial Colony)

Subsequently, a method for acquiring the coordinate of the bacterialcolony will be described. FIG. 19 is a plane schematic diagram showingone example of the image of the bacterial colony in the petri dish. Whenthe microorganism is included in the specimen, consecutive non-isolatedbacterial colonies 1901 are verified in the culturing medium, afterculturing. From the image of the petri dish shown in FIG. 19, theoriginal point of the coordinate is defined by image processing and thecoordinate of the bacterial colony is acquired. One example of an imageprocessing method will be described below, but the present invention isnot limited thereto.

An outer periphery (circle) of the petri dish is acquired by imageprocessing and a central position and a radius thereof are acquired. Thebacterial colony within the outer periphery of the petri dish isextracted by contour detection. In the contour detection, a filter thatdetects a contour, such as a high pass filter, a sobel filter, or thelike may be used. A bacterial colony having a diameter in the range of0.1 mm to 2 mm is selected from the contour of the acquired bacterialcolony, and further, a bacterial colony positioned at a leftmost side ofa screen is called a bacterial colony A1902 and a bacterial colonypositioned at a rightmost side is called a bacterial colony B1903, andthe center of an area within each contour is acquired. The center of thebacterial colony A1902 is set as the original point of the coordinateand a direction toward the center of the bacterial colony B1903 from theoriginal point of the coordinate is defined as an X coordinate axis1904. A Y coordinate axis 1905 is defined in a direction to rotate in acounterclockwise direction from the X coordinate axis 1904 by 90°. Afterthe coordinate axes are determined, the center of each bacterial colonyin the screen is acquired and respective coordinates are acquired andwritten in the control unit 1820 together with the identification codeand the petri dish image of the petri dish.

Subsequently, another method will be described. FIG. 20A is a planeschematic diagram showing one example of the image of the bacterialcolony in the petri dish. Cross-type markers C2001 and D2002 which arealignment targets are added to the petri dish in advance. As describedin FIG. 19, the markers are also detected by the contour detection ofthe bacteria colony. FIG. 20B is a partially enlarged plan schematicdiagram of neighboring regions of the markers of FIG. 20A. A centerpoint of a cross of the marker C2001 is called a starting point 2003 andset as the original point of the coordinate. A center point of a crossof the marker D2002 is called an ending point 2004. A line that linksthe starting point 2003 of the marker C and the ending point 2004 of themarker D is called an X coordinate axis 2005 and a Y coordinate axis2006 which is perpendicular thereto is defined. After the coordinateaxes are determined, the center of each bacterial colony in the screenis acquired and respective coordinates are acquired and written in thecontrol unit 1820 together with the identification code and the petridish image of the petri dish.

A shape of the marker is not particularly limited so long as one pointthereof may be primarily defined. A shape such as a cross shape, arectangular shape, a triangular shape, or an I shape may be used.Further, the marker may be printed directly in the petri dish andstamped in the petri dish. In manufacturing the petri dish, a pattern ofthe marker may be installed in a frame for resin-molding the petri dishand the petri dish having the marker may be fabricated. The marker maybe attached to the petri dish by using a seal. Further, when the markeris fabricated on the bottom of the petri dish, the bacterial colony isgrown on a medium having a thickness, and as a result, when the heightsof the marker and the bacterial colony are different from each other,there is a possibility that both sides cannot be focused simultaneously.In the case of the imaging system considering a focus depth, the seal ofthe marker made of a resin film may be attached onto the surface of themedium. Since the original point and the coordinate are defined afterimaging, positional precision of the marker is available.

The marker may be installed in the petri dish before being inputted intothe harvesting device of the bacteria colony and may be installed insidethe device.

(Selection Means)

Subsequently, a selection means of a bacteria colony to be harvestedwill be described. In the imaging stage 1803, the image of the bacterialcolony and the coordinate of the bacterial colony are stored in thestorage means within the control unit 1820 and thereafter, an operatorreviews the images. The operator views the bacterial colony displayed onthe screen and selects the bacterial colony to be harvested. As a methodfor the selection, for example, the bacterial colony may be selected bymoving a mouse cursor on the screen and may be selected by displayingthe bacterial colony on a touch screen and touching the bacterial colonyon the screen. After the operator selects the bacterial colony, theharvesting device of the bacterial colony performs contour detection ofthe bacterial colony and stores the center of the contour as thecoordinate of the bacterial colony to be harvested through the controlunit 1820.

(Comparison Means)

Subsequently, in the harvesting stage, a method for detecting andharvesting the bacterial colony selected by the operator will bedescribed. When the petri dish is transported to the harvesting stage,the image of the petri dish 1800 is acquired by the image displayingmeans 1809 and displayed on the image outputting device 1821. Further,the identifier of the petri dish 1800 is verified and the imaged imageof the petri dish stored in the storage means within the control unit1820 is read to the image outputting device 1821. The contour detectionof the image of the petri dish acquired by the image displaying means isperformed and the imaged image and the displayed image are compared witheach other by the comparison means within the image outputting device1821. When the petri dish is set in the buffer station or transported bythe transportation means, the position thereof is deviated, such thatthe imaged image and the displayed image may be misaligned in arotational direction. When the bacterial colony is used as the originalpoint of the coordinate, the original point of the coordinate may not beacquired through the same rule with respect to the imaged image and thedisplayed image which is inclined at a predetermined rotational angle.The reason is that as the displayed image rotates, the bacterial colonyat the leftmost side and the bacterial colony at the rightmost side aredifferent from each other. When the bacterial colony is set as theoriginal point of the coordinate, the displayed image is virtuallyrotated on the memory several times while the center of the petri dishacquired from the contour of the petri dish is set as the rotationalcenter. In addition, a rotational position of the displayed image ismatched by pattern matching and the original point of the coordinate iscalculated in that state. When the original point of the coordinate isdetermined, the coordinate of the bacterial colony which is stored isread and the harvesting tool 1810 including the driving unit moves tothe position of the coordinate of the bacterial colony from a newlydetermined original point of the coordinate to harvest the bacteria.

Coordinate correction or correction of distortion of the coordinate suchas parallel movement, enlargement, contraction, and the like may beperformed with respect to the displayed image in addition to rotatablemovement by the comparison means.

Further, a marker for correcting the rotational angle may be installedin the petri dish instead of calculating a correlation with the image ofthe petri dish by virtually rotating the displayed image. FIG. 21 is across-sectional schematic diagram showing a petri dish, a rotationalstage, and a photo sensor. A rotational-direction alignment marker 2103is installed on a side 2101 of the petri dish. The marker may beinstalled by methods such as, for example, printing, stamping, molding,attachment, and the like of a line-shaped record. The imaging stage hasa rotational stage 2102 on the same axis as the center of the petri dishand a photo sensor 2104 detecting the marker 2103 on the side isinstalled at a circumferential portion thereof. After the petri dish isset in the stage, the stage stops at a position where the photo sensor2104 detects the marker 2103 by rotating the rotational stage 2102 andimages the image. Similarly, the harvesting stage also includes therotational stage and the photo sensor and a relationship between aposition where the photo sensor is detected and the position of thecamera coincides with the position of the imaging stage. When the stagestops at the position where the photo sensor detects the marker 2103 byrotating the petri dish on the stage and images the image, the imagedimage and the displayed image which are not misaligned in the rotationaldirection may be acquired.

(Harvesting Means)

As shown in FIG. 18, the harvesting tool moves on a rail 1811 to moveonto the bacterial colony where the petri dish is harvested to harvestthe bacteria and the petri dish moves directly below the harvesting toolto harvest the bacteria.

According to the harvesting device of the bacterial colony according tothe present invention, harvesting may start sequentially from the petridish of which imaging is completed in the imaging stage. Accordingly,since imaging and harvesting may be performed in parallel, throughputcan be improved.

FIG. 22 is a schematic diagram showing another example of a device forharvesting a bacterial colony according to the present embodiment. In aconfiguration of the device of FIG. 22, the imaging stage and theharvesting stage are placed in a common area. Similarly as the case ofFIG. 18, they may be independently operated. The image is imaged anddisplayed by using an imaging and displaying camera 2204. A control unit2220 controls an imaging and harvesting stage 2203, a first bufferstation 2201, a second buffer station 2208, a first transportation means2202, a second transportation means 2207, and the imaging and displayingcamera. Further, the control unit 2220 includes the storage meansstoring the imaged image, the selection means selecting the bacterialcolony to be harvested from the imaged image, and the comparison meanscomparing the imaged image and the displayed image.

In the harvesting device of the bacterial colony of FIG. 22, since thepetri dish of which imaging is completed or the petri dish of whichharvesting is completed may be stored in the buffer station, analysisefficiency can be increased. For example, imaging is performed duringbreak or night when the operator is not present and the petri dish afterimaging is stored in the buffer station. The operator selects thebacterial colony to be harvested by using the selection means after thebreak is completed or a work of the next day starts. Harvesting startssequentially from the petri dish in which the bacterial colony isselected and the petri dish of which harvesting is completed is alsostored in the buffer station. When the device shown in FIG. 22 is usedas described above, analysis throughput can be improved because a waittime of the work is not made. Further, since the imaging stage and theharvesting stage are integrated with each other, the device may besimply configured as compared with the device of FIG. 18 and an effectof miniaturizing the device and reducing cost can be acquired.

As described above, according to the harvesting device of the bacterialcolony according to the present invention, since imaging and harvestingthe image may be processed in parallel, analysis can be efficientlyperformed without waiting time for processing even when hundreds ofspecimens are processed.

Fourth Embodiment

FIG. 23 shows one example of a device for harvesting a bacterial colonyin a fourth embodiment. Prior to using the device in the presentembodiment, specimens such as sputum, urine, pus, and the like collectedfrom a patient are applied to the agar medium and separately cultured,and the bacterial colony is formed. A predetermined amount of culturesolution in a blood culture vessel, which is judged to be positive iscollected and inoculated into a new culture medium and after culturing,the bacterial colony may be formed. In most cases, the bacterial colonymay be acquired through culturing for one day (approximately 12 to 18hours). Reference numeral 2301 represents a petri dish supply stacker,reference numeral 2302 represents a bacterial colony imaging unit,reference numeral 2303 represents an image processing unit, referencenumeral 2304 represents a bacterial colony harvesting unit, referencenumeral 2305 represents a bacterial liquid preparing unit, and referencenumeral 2306 represents a discharge stacker. The petri dish is set inthe petri dish supply stacker 2301 where the bacterial colony is grown,transported to the imaging unit 2302 by the transportation means 2307,and set at predetermined positions suitable for imaging with respect toillumination units 2308, 2309, and 2311, and a camera 2310.

An embodiment of illumination placement of the bacterial colony imagingunit 2302 is shown in FIG. 24. Reference numeral 2408 represents alow-angle illumination unit, reference numeral 2409 represents ahigh-angle illumination unit, reference numeral 2410 represents acamera, and reference numeral 2411 represents a transmitted illuminationunit. Reference numeral 2412 represents a light shielding plate,reference numeral 2413 represents a base, and reference numeral 2414represents a set petri dish. Referring back to FIG. 23, the bacterialcolony acquired from the specimen is grown on the surface of the agarmedium in the petri dish 2314.

The low-angle illumination unit 2308, the high-angle illumination unit2309, and the transmitted illumination unit 2311 are connected to anillumination control unit 2315 and may illuminate the petri dish througha predetermined combination. When the transmitted illumination unit 2311is lit on, illumination light is set to illuminate the petri dish 2314,and further, when the low-angle illumination unit 2308 and thehigh-angle illumination unit 2309 are lit on, a bottom surface of thepetri dish 2314 is darkened. The bacterial colony on the petri dish 2314is imaged by the camera 2310 and the acquired image is transmitted tothe image input means 2316. Reference numeral 2317 represents an imageprocessing means, which extracts an area of the bacterial colony byprocessing image data transmitted to the image input means 2316, andfurther, an image feature amount from each extracted bacterial colony iscalculated by using the image processing means 2317. The image featureamount includes, for example, a boundary length, an area, colorinformation, luminosity information, and a difference from backgroundluminosity of the bacterial colony area. Reference numeral 2318represents a secondary storage device and may store the imaged image,the processed image, data, and the position and the area of thebacterial colony. The image processing means 2317 analyzes the acquiredfeature amount and performs grouping of a plurality of kinds ofbacterial colonies on the petri dish based on a past database receivedin the secondary storage device 2318.

The petri dish after imaging is transported to the bacterial colonyharvesting unit 2304 by a transportation means 2319. Reference numeral2320 represents a harvesting needle, reference numeral 2321 represents aZ stage, and reference numeral 2322 represents an XY stage. Inputtedimage information is projected on a monitor 2323 and a laboratorytechnician selects and determines a plurality of bacterial colonies tobe harvested. Determination information is transferred to the bacterialcolony harvesting unit 2304 together with the position of the bacterialcolony. By controlling the Z stage 2321 and the XY stage 2322, theharvesting needle 2320 is moved to the coordinate of the bacteria colonyof the petri dish 2314 to pick up the bacterial colony.

FIG. 24 shows a detailed configuration diagram of the imaging unit ofFIG. 23. The low-angle illumination unit 2308 is constituted bylow-angle illumination units 2411, 2412, 2413, and 2414 and light-onthereof may be individually controlled by using an illumination controlunit 2315. The high-angle illumination unit 2309 is constituted byhigh-angle illumination units 2421, 2422, 2423, and 2424 and light-onthereof may also be individually controlled. Since the surface of thebacterial colony is comparatively smooth, a spot subjected to directreflection is imaged brightly by adjusting the illumination.

The illumination is lit on for each direction to acquire luminosities inthe illumination, at the center of the bacterial colony, and on asurface including the optical axis of the camera. In acquiring adetailed shape of the bacterial colony, it is preferable to combineseveral kinds of images by imaging among those of four types of imagesilluminated by the low-angle illumination units 2411, 2412, 2413, and2414, respectively one by one, four types of images illuminated by thehigh-angle illumination units 2421, 2422, 2423, and 2424, respectivelyone by one, the image illuminated by the transmitted illumination unit2311 and images which are imaged by the illumination from pluraldirections simultaneously.

In the high-angle illumination, according to the position of reflectionlight, an angle of the bacterial colony at the position of thereflection light may be acquired. The normal-line direction of thebacterial colony at the position of the bacterial colony acquiring thedirectly reflected light is expressed by (Vc+Vi)/2 by using a unitvector Vc from the position of the reflection light toward the cameralens and a unit vector Vi from the position of the reflection light tothe illumination. When the bacterial colony is flat, a position wherethe reflected light is detected may be estimated to be around the centerof the bacterial colony and when the position of the reflected light isaround the bacterial colony, the bacterial colony may be estimated tohave a height from the culture medium. Further, when the bacterialcolony has a shape collapsed from a dome shape, the reflected light fromit is detected at a plurality of positions. After the bacterial colonyis extracted, the image feature amount is calculated for each bacterialcolony. Feature amounts such as the size, color, surface irregularity,shape, or the like of the area of the detected bacterial colony areextracted by image processing using the high-angle illumination image(illuminated by 2309), the low-angle illumination image (illuminated by2308), and the transmitted illumination image (illuminated by 2311).Further, the circularity is acquired by using the long axis/short axiswhen the shape is elliptically approximated. The luminosity, color,transmittance, and cubic shape of the bacterial colony are acquired fromthe transmitted illumination image. The transmittance and the heightfrom the culture medium may be acquired based on the luminosity anddirect reflection of the high-angle illumination image.

The transmitted illumination may be used only when the culture mediumtransmits light, and in this case, the transmitted illumination becomesdark at the position of the bacterial colony, and the thickness of thebacterial colony increases, and further, becomes dark. When anyluminosity in the bacterial colony is set as IC(x, y) and an averagevalue of the luminosity of the medium is set as IM, the thickness D(x,y) of the colony at a position of (x, y) is acquired by an equation (Eq.7) below.

D(x,y)=−G(log IC(x,y)−log IM)  (Eq. 7)

Herein, G represents a gain determined for each bacterial colonyspecies.

It is assumed that the reflected light by the high-angle illuminationcan be detected at a position (X, Y). In this case, it is assumed thatthe normal vector is inclined from the vertical direction by θ. In thiscase, a change in thickness when the position is deviated by Δx becomes−Δx tan θ. Herein, when a difference of D(x, y) is calculated, anequation below is established.

D(X+Δx,Y)−D(X,Y)=−G (log IC(X+Δx,Y)−log IC(X,Y))=−Δx tan θ(Eq. 8)

G=Δx tan θ/(log IC(X+Δx,Y)−log IC(X,Y))  (Eq. 9)

That is, G is acquired by the equation above, and as a result, thethickness of the bacterial colony is acquired at any position. G shownby (Eq. 9) is constant in the same bacterial colony specimen(alternatively, bacterial colony). G is acquired in the place wheredirect reflection of the high-angle illumination is detected. The G maybe used even in a place where direct reflection cannot be detected. Thecubic shape is determined by acquiring the height. The height isdetermined by luminosity and G at (x, y) acquired by the transmittedillumination image. G is determined by information of direct reflectionin the high-angle illumination image. Since G is constant in the samebacterial colony species (alternatively, bacterial colony), only onepoint at any reflected point (x, y) is sufficient.

Accordingly, in the case of a medium having comparatively hightransmittance, when only a place where directly reflected light isgenerated is discovered, the cubic shape and volume of the bacterialcolony may be acquired. The volume may be acquired by integrating theacquired thickness (height) with an area.

As described above, logically, in order to acquire the volume, thevolume may be calculated only by the high-angle illumination and thetransmitted illumination and the low-angle illumination is notnecessarily required. However, in the high-angle illumination, directreflection causes the shape of the bacterial colony not to be known insome bacterial colonies (in case the periphery and the edge of thebacterial colony are reflected). In this case, the low-angleillumination may be used instead of the high-angle illumination.

FIG. 25 shows a relationship between the volume (mm³) of the bacterialcolony for each bacterial species and turbidity (McF). It can be seenthat turbidity of suspended bacterial liquid is different even in thesame volume for each bacterial species. It is considered as one of thereasons that this difference is caused due to a difference of a contentof polysaccharide such as mucoid configuring a cell wall for eachbacterium. According to FIG. 25, in the case where 3M1 of a bacterialliquid having McF of 0.6 is prepared, when at least the bacterial colonyof 1 mm³ or more is collected from a total amount, the bacterial liquidis prepared with higher concentration than McF of 0.6 in almostbacterial species. When 1 mm³ is divided by the calculated volume of thebacterial colony according to the method of the present invention, theminimum number of bacterial colonies required to prepare the bacterialliquid may be acquired. Meanwhile, since turbidity per predeterminedvolume is different for each bacterial species as shown in FIG. 25, whenthe bacterial species of the bacterial colony is known, the number ofbacterial colonies required to prepare the bacterial liquid may becalculated more accurately. However, in general, during bacteriatesting, the bacterial species is not identified at this time and a nameof a bacterial species may not be known in harvesting. Herein, in thepresent invention, as described above, the image of the bacterial colonyis analyzed in illumination in a plurality of directions, the featureamount is extracted, the bacterial colonies are grouped by a databasewhich is stored in advance, and a kind of the bacterial colony isestimated. When grouping is accomplished, turbidity per predeterminedvolume may be known from the database and the number of harvestedbacteria required to prepare the bacterial liquid may be acquired moreaccurately by considering the estimated volume of the bacterial colonyacquired from the image and a coefficient for each bacterial species.For reference, the number of harvested bacteria required to actuallyacquire predetermined turbidity is shown in FIG. 26.

When grouping is impossible, the number of harvested bacteria may becalculated with a fixation value according to a bacterial species havingthe lowest turbidity. As described above, the volume of each bacterialcolony is acquired by the image processing means 2317, and further, thenumber of bacterial colonies required to acquire a predeterminedconcentration of bacterial liquid is calculated. The information istransferred to the bacterial colony harvesting unit 2304. The bacterialcolony harvesting unit 2304, according to the positional information ofthe bacterial colony to be harvested and information on the requirednumber of bacterial colonies, harvests the plurality of bacterialcolonies and suspends the harvested bacterial colonies to the normalsaline in a test tube 2324. The turbidity of the suspended bacterialiquid is measured by a turbidimeter (not shown), the required amount ofdiluted solutions is calculated, the normal saline as the dilutedsolution is added, the turbidity is verified again, and when theturbidity is predetermined turbidity, the test tube is discharged.

Meanwhile, when accurate thickness information of the bacterial colonyis not acquired, thickness information prescribed as several degrees(high, medium, and low) according to an area of the bacterial colony isstored in the secondary storage device 2318 in advance and the thicknessinformation may be automatically corresponded according to the area ofthe bacterial colony acquired by the image processing means 2317 basedon the imaged image and an approximate volume may be estimated. Thethickness information to be stored may be set for each bacterial speciesor may have a predetermined value according to a feature of the shape ofthe bacterial colony by the bacterial species. When grouping succeeds,accurate thickness information unique to the bacterial species is usedand the volume of the bacterial colony may be estimated by multiplyingthe thickness information by the area of the bacterial colony calculatedfrom the imaged image. When grouping is not performed, a predeterminedvalue depending on the area, which does not depend on the bacterialspecies is extracted from the secondary storage device 3218 and used toacquire the approximate volume. Further, the thickness information isnot automatically calculated according to the area, but a user may inputinformation from the image processing unit 2030 or select and designatethe information from candidates of several degrees.

FIG. 27 shows an example of a screen for input. A thickness input area2702 is present on a monitor 2323 in addition to a bacterial colonyimage area 2701. The image of the bacterial colony on the petri dishpicked up by the camera 2310 is processed, and herein, the bacterialcolonies are classified into three groups of 2703 (group α), 2704 (groupβ), and 2705 (group γ) and are displayed on the bacterial colony imagearea. The user inputs the thicknesses such as an end thickness (A) of0.1 mm, a middle portion thickness (B) of 0.15 mm, and a center portionthickness (C) of 0.2 mm with respect to, for example, the bacterialcolony of group α of 2703. The input is performed for each group of thebacterial colony to be harvested. Alternatively, the values of A, B, andC may have options, respectively, and may be selected.

FIG. 28 shows an example of another screen. The thickness of one spot ofthe bacterial colony may be inputted or selected. The user's input isfurther simplified and any one of low, medium and high is selected inthe thickness input area 2802. Herein, an example of selecting medium isshown. The method for calculating the number of bacterial colonies to beharvested from the relationship between the volume and the turbidityafter calculating the volume is the same as the process when the heightinformation is accurately acquired. Further, when a predeterminedbacterial species is assumed and thus grouping by the feature amount ofthe bacterial colony is not required, like a case in which the method isused in an industrial field, a combination of the plurality ofilluminations of the low-angle illumination unit 2308, the high-angleillumination unit 2309, and the transmitted illumination unit 2311 isnot required, the configuration of the device is simplified, and amanufacturing cost can also be reduced. In this case, the area may becalculated from the imaged image by the image processing means 2317,predetermined thickness information may be input or selected from theplurality of candidates, and the approximate volume may be acquired bymultiplying the thickness information by the area.

INDUSTRIAL APPLICABILITY

The present invention can be applied to a device for harvesting abacterial colony that cultures bacteria in a specimen, selects abacteria colony, and adjusts a bacterial liquid of the selectedbacterial colony in order to identify bacteria included in the specimenand test drug susceptibility.

REFERENCE SIGNS LIST

-   -   101 . . . Low-angle illumination unit    -   102 . . . High-angle illumination unit    -   103 . . . Camera    -   104 . . . Transmitted illumination unit    -   105 . . . Light shielding plate    -   106 . . . Base    -   107 . . . Petri dish    -   108 . . . Petri dish transportation means    -   109 . . . Stacker    -   110 . . . Petri dish transportation means    -   111 . . . Harvesting needle    -   112 . . . Z stage    -   113 . . . XY stage    -   114 . . . Petri dish    -   115 . . . Light control unit    -   116 . . . Stage control unit    -   117 . . . Image input means    -   118 . . . Grouping means    -   119 . . . Image processing means    -   120 . . . Transmitted illumination unit    -   121 . . . upper illumination unit    -   122 . . . Camera    -   123 . . . Micro plate    -   124 . . . GUI    -   125 . . . Secondary storage device    -   130 . . . Input/output means    -   140 . . . Entire control unit    -   1800 . . . Petri dish    -   1801 . . . First buffer station    -   1802 . . . First transportation means    -   1803 . . . Imaging stage    -   1804 . . . Image imaging camera    -   1805 . . . Second transportation means    -   1806 . . . Second buffer station    -   1807 . . . Third transportation means    -   1808 . . . Harvesting stage    -   1809 . . . Image display means    -   1810 . . . Harvesting tool    -   1811 . . . Rail    -   1812 . . . Test tube    -   1813 . . . Bacterial liquid concentration adjusting system    -   1814 . . . Suspension discharging system    -   1815 . . . Suspension rack    -   1820 . . . Control unit    -   1821 . . . Image outputting device    -   1901 . . . Non-isolated bacterial colony    -   1902 . . . Bacterial colony A    -   1903 . . . Bacterial colony B    -   2102 . . . Rotational stage    -   2103 . . . Marker    -   2104 . . . Photo sensor    -   2200 . . . Petri dish    -   2201 . . . First buffer station    -   2202 . . . First transportation means    -   2203 . . . Imaging and harvesting stage    -   2204 . . . Image imaging and displaying camera    -   2205 . . . Test tube    -   2206 . . . Bacterial liquid concentration adjusting system    -   2207 . . . Second transportation means    -   2208 . . . Second buffer station    -   2209 . . . Third transportation means    -   2210 . . . Suspension discharging system    -   2211 . . . Suspension rack    -   2220 . . . Control unit    -   2301 . . . Petri dish supply stacker    -   2302 . . . Imaging unit (bacterial colony imaging unit)    -   2303 . . . Image processing unit    -   2304 . . . Bacterial colony harvesting unit    -   2305 . . . Bacterial liquid preparing unit    -   2306 . . . Discharge stacker    -   2307, 2319 . . . Transportation means    -   2308 . . . High-angle illumination unit    -   2309 . . . Low-angle illumination unit    -   2310 . . . Camera    -   2311 . . . Transmitted illumination unit    -   2312 . . . Light shielding plate    -   2313 . . . Base    -   2314 . . . Petri dish    -   2315 . . . Illumination control unit    -   2316 . . . Image input means    -   2317 . . . Image processing means    -   2318 . . . Secondary storage device    -   2320 . . . Harvesting needle    -   2321 . . . Z stage    -   2322 . . . XY stage    -   2323 . . . Monitor    -   2324 . . . Test tube    -   2701 . . . Bacterial colony image area    -   2702 . . . Thickness input area    -   303 . . . Group α    -   304 . . . Group β    -   305 . . . Group γ    -   2411, 2412, 2413, 2414 . . . Low-angle illumination unit    -   2421, 2422, 2423, 2424 . . . High-angle illumination unit

1. A device for harvesting a bacterial colony, comprising: an upperillumination unit illuminating a bacteria colony cultured on a culturemedium received in a first vessel which is optically transparent, fromthe top; a transmitted illumination unit illuminating the bacterialcolony by transmitting the optically transparent vessel and the culturemedium with illumination light; an imaging unit imaging the bacterialcolony sequentially illuminated by the upper illumination unit and thetransmitted illumination unit; an image processing unit extracting animage of a bacterial colony to be harvested by processing an image ofthe bacterial colony illuminated by the upper illumination unit and animage of the bacterial colony illuminated by the transmittedillumination unit, which are imaged by the imaging unit; and aharvesting unit harvesting the bacterial colony extracted by the imageprocessing unit from the culture medium and moving the harvestedbacterial colony to a second vessel.
 2. The device for harvesting abacterial colony according to claim 1, further comprising: a displayunit having a display screen, wherein a result of classifying thebacterial colony by processing the image of the bacterial colonyilluminated by the upper illumination unit and the image of thebacterial colony illuminated by the transmitted illumination unit in theimaging processing unit is displayed on the screen of the display unit.3. The device for harvesting a bacterial colony according to claim 2,wherein the image display unit inputs the classification result modifiedon the screen on which the result of classifying the bacterial colony isdisplayed into the image processing unit.
 4. The device for harvesting abacterial colony according to claim 1, wherein the image processing unitseparates and detects the image of the bacterial colony from an image ofa character or a symbol written or formed on the optically transparentvessel.
 5. The device for harvesting a bacterial colony according toclaim 1, wherein a gain determined for each species of the bacterialcolony is acquired by processing with the imaging unit the imageacquired by imaging the bacterial colony illuminated by the upperillumination unit with the imaging unit, luminosity of the culturemedium is acquired by processing with the image processing unit theimage acquired by imaging the bacterial colony illuminated by thetransmitted illumination unit with the imaging unit, and the thicknessof the bacterial colony is acquired from the acquired gain andluminosity of the culture medium.
 6. The device for harvesting abacterial colony according to claim 1, wherein the upper illuminationunit includes a high-angle illumination unit illuminating the bacterialcolony from a high-angle direction and a low-angle illumination unitilluminating the bacterial colony from a low-angle direction.
 7. Thedevice for harvesting a bacterial colony according to claim 1, furthercomprising: an analysis unit performing infrared spectroscopy or Ramanspectroscopy with respect to the harvested bacteria colony, wherein agrouping result of the harvested bacterial colony is capable of beingverified again by the analysis unit.
 8. The device for harvesting abacterial colony according to claim 1, wherein the harvesting unitincludes a camera imaging the bacterial colony extracted with the imageprocessing unit and the image of the bacterial colony imaged with thecamera is compared with the image of the bacterial colony imaged withthe imaging unit to specify the bacterial colony to be harvested.
 9. Thedevice for harvesting a bacterial colony according to claim 1, furthercomprising: a harvested bacteria number calculating unit calculating thenumber of harvested bacteria required to prepare a bacterial liquidhaving a predetermined concentration by using thickness information ofthe bacterial colony of several steps prepared in advance according tothe size of an area of the bacterial colony from the image of thebacterial colony extracted with the image processing unit.
 10. Thedevice for harvesting a bacterial colony according to claim 9, whereinthe thickness information of the bacterial colony is capable of beingset by a user.
 11. A method for harvesting a bacterial colony,comprising: illuminating a bacteria colony cultured on a culture mediumreceived in a first vessel which is optically transparent, from upperside and imaging the bacterial colony to acquire an upper sideillumination image of the bacterial colony; illuminating the bacterialcolony by transmitting the optically transparent vessel and the culturemedium with illumination light and imaging the bacterial colony toacquire a transmitted light illumination image of the bacterial colony;extracting an image of a bacterial colony to be harvested by processingthe upper side illumination image and the transmitted light illuminationimage; and harvesting a bacterial colony corresponding to the extractedimage from the culture medium and moving the harvested bacterial colonyto a second vessel.
 12. The method for harvesting a bacterial colonyaccording to claim 11, wherein a result of classifying the bacterialcolony by processing the upper side illumination image and thetransmitted light illumination image of the bacterial colony isdisplayed on a screen.
 13. The method for harvesting a bacterial colonyaccording to claim 12, wherein the classification result is modified onthe screen on which the result of classifying the bacterial colony isdisplayed.
 14. The method for harvesting a bacterial colony according toclaim 11, wherein in the extracting of the image of the bacterial colonyto be harvested, the image of the bacterial colony is separated anddetected from an image of a character or a symbol written or formed onthe optically transparent vessel.
 15. The method for harvesting abacterial colony according to claim 11, wherein in the acquiring of theupper side illumination image of the bacterial colony, a high-angleillumination image acquired by illuminating the bacterial colony from ahigh-angle direction and a low-angle illumination image acquired byilluminating the bacterial colony from a low-angle direction areacquired.
 16. The method for harvesting a bacterial colony according toclaim 11, wherein a gain determined for each species of the bacterialcolony is acquired by processing the upper side illumination image,luminosity of the culture medium is acquired by processing thetransmitted light illumination image, and the thickness of the bacterialcolony is acquired from the acquired gain and luminosity of the culturemedium.
 17. The method for harvesting a bacterial colony according toclaim 11, wherein infrared spectroscopy or Raman spectroscopy isperformed with respect to the harvested bacteria colony and theclassification result of the harvested bacterial colony is verifiedagain.
 18. The method for harvesting a bacterial colony according toclaim 11, wherein the bacterial colony harvested and moved to the secondvessel is imaged with a camera to acquire the image of the bacterialcolony and the acquired image is compared with an image of a bacterialcolony to be harvested, which is extracted by processing the upper sideillumination image and the transmitted light illumination image, tospecify the bacterial colony to be harvested.
 19. The method forharvesting a bacterial colony according to claim 11, wherein the numberof harvested bacteria required to prepare a bacterial liquid having apredetermined concentration is calculated by using thickness informationof the bacterial colony of several steps prepared in advance accordingto the size of an area of the bacterial colony from the image of thebacterial colony of the upper imaged image.
 20. The method forharvesting a bacterial colony according to claim 19, wherein thethickness information of the bacterial colony is capable of being set bya user.
 21. A device for harvesting a bacterial colony, comprising: aspecimen stage on which a petri dish is loaded; an imaging stage imagingthe inside of the petri dish from a top surface of the petri dish andhaving a storage unit storing the image; a selection unit automaticallydetecting bacterial colonies of microorganisms in the petri dish andselecting some of the detected bacterial colonies; an image display unitdisplaying the image in the petri dish from the top surface of the petridish; a comparison unit reading the image imaged with the imaging stagefrom the storage unit and comparing the read image with the imagedisplayed with the image display unit; a harvesting unit including adriving unit for harvesting the bacterial colonies of the microorganismsin the petri dish; a harvesting stage including the harvesting unit, theimage display unit, the comparison unit, and the selection unit; a firstbuffer station storing a plurality of petri dishes before imaging; asecond buffer station storing a plurality of petri dishes after imaging;a first transportation unit sequentially supplying petri dishes beforeimaging from the first buffer station to the imaging stage; a secondtransportation unit sequentially supplying petri dishes after imagingfrom the imaging stage to the second buffer station; a thirdtransportation unit sequentially supplying petri dishes beforeharvesting from the second buffer station to the harvesting stage; and acontrol unit controlling the imaging stage, the harvesting stage, thefirst buffer station, the second buffer station, the firsttransportation unit, the second transportation unit, and the thirdtransportation unit, wherein the selection unit automatically selects abacterial body to be harvested from an appearance feature of thebacterial colony in the imaged petri dish with respect to each petridish.
 22. The device for harvesting a bacterial colony according toclaim 21, wherein the imaging stage and the harvesting stage areoperable independently from each other.
 23. The device for harvesting abacterial colony according to claim 21, wherein the petri dish has amarker for specifying an original point of a coordinate.
 24. A devicefor harvesting a bacterial colony, comprising: a specimen stage on whicha petri dish is loaded; an imaging stage imaging the inside of the petridish from a top surface of the petri dish and having a storage unitstoring the image; a selection unit automatically detecting bacterialcolonies of microorganisms in the petri dish and selecting some of thedetected bacterial colonies; an image display unit displaying the imagein the petri dish from the top surface of the petri dish; a comparisonunit reading the image imaged with the imaging stage from the storageunit and comparing the read image with the image displayed with theimage display unit; a harvesting unit including a driving unit forharvesting the bacterial colonies of the microorganisms in the petridish; a harvesting stage including the harvesting unit, the imagedisplay unit, the comparison unit, and the selection unit; a firstbuffer station storing a plurality of petri dishes before imaging; asecond buffer station storing a plurality of petri dishes after imaging;a first transportation unit sequentially supplying petri dishes beforeimaging from the first buffer station to the imaging stage; a secondtransportation unit sequentially supplying petri dishes after imagingfrom the imaging stage to the second buffer station; and a control unitcontrolling the imaging stage, the harvesting stage, the first bufferstation, the second buffer station, the first transportation unit, andthe second transportation unit, wherein the imaging stage and theharvesting stage are positioned within a common area.
 25. The device forharvesting a bacterial colony according to claim 24, wherein the imagingstage and the harvesting stage are operable independently from eachother.
 26. The device for harvesting a bacterial colony according toclaim 24, wherein the selection unit sets an original point of acoordinate and a coordinate axis of a petri dish image while imaging thebacterial colony and stores a coordinate of a bacteria colony to beharvested.
 27. The device for harvesting a bacterial colony according toclaim 24, wherein the petri dish has a marker for specifying an originalpoint of a coordinate.
 28. The device for harvesting a bacterial colonyaccording to claim 24, wherein the first buffer station and the secondbuffer station are commonly used.